Impact of STROBE Statement Publication on Quality of Observational Study Reporting: Interrupted Time Series versus Before-After Analysis
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
BACKGROUND: In uncontrolled before-after studies, CONSORT was shown to improve the reporting of randomised trials. Before-after studies ignore underlying secular trends and may overestimate the impact of interventions. Our aim was to assess the impact of the 2007 STROBE statement publication on the quality of observational study reporting, using both uncontrolled before-after analyses and interrupted time series. METHODS: For this quasi-experimental study, original articles reporting cohort, case-control, and cross-sectional studies published between 2004 and 2010 in the four dermatological journals having the highest 5-year impact factors (≥ 4) were selected. We compared the proportions of STROBE items (STROBE score) adequately reported in each article during three periods, two pre STROBE period (2004-2005 and 2006-2007) and one post STROBE period (2008-2010). Segmented regression analysis of interrupted time series was also performed. RESULTS: Of the 456 included articles, 187 (41%) reported cohort studies, 166 (36.4%) cross-sectional studies, and 103 (22.6%) case-control studies. The median STROBE score was 57% (range, 18%-98%). Before-after analysis evidenced significant STROBE score increases between the two pre-STROBE periods and between the earliest pre-STROBE period and the post-STROBE period (median score2004-05 48% versus median score2008-10 58%, p<0.001) but not between the immediate pre-STROBE period and the post-STROBE period (median score2006-07 58% versus median score2008-10 58%, p = 0.42). In the pre STROBE period, the six-monthly mean STROBE score increased significantly, by 1.19% per six-month period (absolute increase 95%CI, 0.26% to 2.11%, p = 0.016). By segmented analysis, no significant changes in STROBE score trends occurred (-0.40%; 95%CI, -2.20 to 1.41; p = 0.64) in the post STROBE statement publication. INTERPRETATION: The quality of reports increased over time but was not affected by STROBE. Our findings raise concerns about the relevance of uncontrolled before-after analysis for estimating the impact of guidelines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61602-x · Cited by 21,320
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010 · Cited by 17,796
Elm, E. V., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · BMJ · doi:10.1136/bmj.39335.541782.ad · Cited by 10,766
Segmented regression analysis of interrupted time series studies in medication use research
Wagner, A. K., Soumerai, S. B., Zhang, F., & Ross-Degnan, D. (2002) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.1365-2710.2002.00430.x · Cited by 3,279 · cited 3×
“We used a segmented linear regression model to determine the impact of STROBE over time [18,19,20].”
BackgroundMethodvon Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · Preventive Medicine · doi:10.1016/j.ypmed.2007.08.012 · Cited by 2,698
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · Epidemiology · doi:10.1097/ede.0b013e3181577654 · Cited by 1,991
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Bulletin of the World Health Organization · doi:10.2471/blt.07.045120 · Cited by 1,632
Moher, D., Jones, A., Lepage, L., & for the CONSORT Group (2001) · JAMA · doi:10.1001/jama.285.15.1992 · Cited by 1,027 · cited 3×
“Moreover, in a comparison of the quality of reporting of randomised controlled trials in four journals, of which three required the use of CONSORT from 1996 onwards (JAMA, British Medical Journal, The Lancet) and one did not (New England Journal of Medicine), a before-after analysis indicated an improvement in quality…”
BackgroundMethodRamsay, C. R., Matowe, L., Grilli, R., Grimshaw, J. M., & Thomas, R. E. (2003) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462303000576 · Cited by 703
Research designs for studies evaluating the effectiveness of change and improvement strategies
Eccles, M. (2003) · Quality and Safety in Health Care · doi:10.1136/qhc.12.1.47 · Cited by 586
Hopewell, S., Dutton, S., Yu, L.-M., Chan, A.-W., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c723 · Cited by 504 · cited 2×
“According to uncontrolled before-after studies, the 1996 CONsolidated Standards of Reporting Trials (CONSORT) statement improved the reporting of randomised trials [15,16,17].”
MethodQuality of reporting of observational longitudinal research.
Tooth, L. (2005) · American Journal of Epidemiology · doi:10.1093/aje/kwi042 · Cited by 363
Issues in the reporting of epidemiological studies: a survey of recent practice.
Pocock, S. J., Collier, T. J., Dandreo, K. J., de Stavola, B. L., Goldman, M. B., Kalish, L. A., Kasten, L. E., & McCormack, V. A. (2004) · BMJ · doi:10.1136/bmj.38250.571088.55 · Cited by 313
Cobo, E., Cortes, J., Ribera, J. M., Cardellach, F., Selva-O'Callaghan, A., Kostov, B., Garcia, L., Cirugeda, L., Altman, D. G., Gonzalez, J. A., et al. (2011) · BMJ · doi:10.1136/bmj.d6783 · Cited by 290
Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies
Papanikolaou, P. N. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050873 · Cited by 229 · cited 1×
“Observational studies may be better able to detect rare or delayed adverse effects of treatments and to reflect outcomes obtained in everyday practice [1].”
BackgroundMills, E. J., Wu, P., Gagnier, J., & Devereaux, P. J. (2005) · Contemporary Clinical Trials · doi:10.1016/j.cct.2005.02.008 · Cited by 216 · cited 3×
“Moreover, in a comparison of the quality of reporting of randomised controlled trials in four journals, of which three required the use of CONSORT from 1996 onwards (JAMA, British Medical Journal, The Lancet) and one did not (New England Journal of Medicine), a before-after analysis indicated an improvement in quality…”
ResultInfluentialIvers, N. M., Taljaard, M., Dixon, S., Bennett, C., McRae, A., Taleban, J., Skea, Z., Brehaut, J. C., Boruch, R. F., Eccles, M. P., et al. (2011) · BMJ · doi:10.1136/bmj.d5886 · Cited by 193 · cited 1×
“Several studies suggested that using the CONSORT statement might improve the reporting of randomised controlled trials [15,16,17,24].”
MethodStatistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial
Cobo, E., Selva-O'Callagham, A., Ribera, J.-M., Cardellach, F., Dominguez, R., & Vilardell, M. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000332 · Cited by 112 · cited 1×
“A few studies used STROBE to assess the quality of observational study reporting [11,12]; however, the impact of STROBE on the quality of observational study reporting has never been assessed excepted 2 randomized studies assessing the impact of adding the STROBE checklist to conventional review on manuscript quality…”
MethodPoorolajal, J., Cheraghi, Z., Irani, A. D., & Rezaeian, S. (2011) · Epidemiology and Health · doi:10.4178/epih/e2011005 · Cited by 79 · cited 3×
“The few studies assessing the quality of observational study reporting, with the STROBE statement as a reference, identified a number of deficiencies consistent with our findings, including marked inadequacies in reporting the management of missing data [11,12,21,22], confounding [21,22], and sample size [11,21,22].”
Cited by
Stevens, A., Shamseer, L., Weinstein, E., Yazdi, F., Turner, L., Thielman, J., Altman, D. G., Hirst, A., Hoey, J., Palepu, A., et al. (2014) · BMJ · doi:10.1136/bmj.g3804 · Cited by 242
Nicholls, S. G., Quach, P., von Elm, E., Guttmann, A., Moher, D., Petersen, I., Sørensen, H. T., Smeeth, L., Langan, S. M., & Benchimol, E. I. (2015) · PLOS ONE · doi:10.1371/journal.pone.0125620 · Cited by 217
“[16] Reporting guidelines have been demonstrated to improve reporting of studies when they are adopted by the research community and relevant journals, [17–20] although their effectiveness varies[20, 21] and may depend on implementation by journals rather than simple endorsement.”
BackgroundFuller, T., Peters, J., Pearson, M., & Anderson, R. (2014) · American Journal of Public Health · doi:10.2105/ajph.2014.302195 · Cited by 30
“Nevertheless, from the data that were available, our results showed reporting completeness to be within the range of that reported by other studies.(46,47) However, our estimates of reporting completeness were likely to be inflated because of the generous, rather than conservative, coding scheme we adopted when coding…”
Result
Metadata
- Authors
- Bastuji-Garin, S., Sbidian, E., Gaudy-Marqueste, C., Ferrat, E., Roujeau, J.-C., Richard, M.-A., Canoui-Poitrine, F., & on behalf of the European Dermatology Network (EDEN)
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-08-26
- Volume
- 8
- Issue
- 8
- Pages
- e64733
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2044249479
- Cited by
- 101
- References
- 20
- FWCI
- 10.40
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.