Unpublished rating scales: a major source of bias in randomised controlled trials of treatments for schizophrenia
Open accessAbstract
Background: A recent review suggested an association between using unpublished scales in clinical trials and finding significant results. Aims: To determine whether such an association existed in schizophrenia trials. Method: Three hundred trials were randomly selected from the Cochrane Schizophrenia Group's Register. All comparisons between treatment groups and control groups using rating scales were identified. The publication status of each scale was determined and claims of a significant treatment effect were recorded. Results: Trials were more likely to report that a treatment was superior to control when an unpublished scale was used to make the comparison (relative risk 1.37 (95% C11.12–1.68)). This effect increased when a ‘gold-standard’ definition of treatment superiority was applied (RR 1.94 (95% C11.35–2.79)). In non-pharmacological trials, one-third of ‘gold-standard’ claims of treatment superiority would not have been made if published scales had been used. Conclusions: Unpublished scales are a source of bias in schizophrenia trials.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Continuous outcome data were extracted only when the psychometric scale had been previously published in a peer-reviewed journal and had not been modified by the trial authors.
“we extracted continuous data only if the psychometric scale used to measure the outcome of interest had been previously published in a peer-reviewed journal (Marshall 2000), and was not subjected to item, scoring, or other modification by the trial authors.”
From Interventions for self-harm in children and adolescents (2021) · cites this paper as Marshall 2000
Continuous data from rating scales were included only if the measuring instrument had been described in a peer-reviewed journal, and not if it was written or modified by a trialist for a particular trial.
“We included continuous data from rating scales only if the measuring instrument had been described in a peer-reviewed journal (Marshall 2000); and not those written or modified by one of the trialists for a particular trial.”
From Compulsory community and involuntary outpatient treatment for people with severe mental disorders. (2017) · cites this paper as Marshall 2000
Continuous data from rating scales were included only if the measuring instrument had been described in a peer-reviewed journal and was self-report or completed by an independent rater or relative.
“We included continuous data from rating scales only if the measuring instrument had been described in a peer-reviewed journal (Marshall 2000), and the instrument was either self report or completed by an independent rater or relative (not the therapist) (Li 2010).”
From Morita therapy for anxiety disorders in adults (2015) · cites this paper as Marshall 2000
Modified versions of validated instruments might no longer be considered validated, and use of unpublished measurement scales can introduce bias and inflate treatment effect sizes.
“Modified versions of validated instruments might no longer be considered validated, and use of unpublished measurement scales can introduce bias and inflate treatment effect sizes.388”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [388]
Marshall et al. [1] showed that in schizophrenia trials authors were more likely to report that treatment was superior to control when an unpublished instrument was used in the comparison, rather than a published instrument.
“Marshall et al. [1] showed that in schizophrenia trials authors were more likely to report that treatment was superior to control when an unpublished instrument was used in the comparison, rather than a published instrument.”
From Evaluation of the methodological quality of systematic reviews of health status measurement instruments (2009) · cites this paper as [1]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
1 retracted
Assertive community treatment for people with severe mental disorders
Marshall, M., & Lockwood, A. (1998) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001089 · Cited by 589 · see citing works
Self-citationRetractedThe Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · see citing works
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127 · see citing works
Gøtzsche, P. C. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90017-2 · Cited by 339 · see citing works
Reporting on quality of life in randomised controlled trials: bibliographic study
Sanders, C., Egger, M., Donovan, J., Tallon, D., & Frankel, S. (1998) · BMJ · doi:10.1136/bmj.317.7167.1191 · Cited by 304 · see citing works
Flagged first, then by citations
1 of the 6 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883 · see citing works
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251 · see citing works
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9606-8 · Cited by 4,260 · see citing works
“To our knowledge, Marshall [3] is the only one who empirically showed that the results of studies can differ when validated measurement instruments are used compared to studies in which non-validated instruments are used.”
BackgroundMethodInfluentialThe Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835 · see citing works
(2008) · Cited by 3,790 · see citing works
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544 · see citing works
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2012) · International Journal of Surgery · doi:10.1016/j.ijsu.2011.10.001 · Cited by 2,345 · see citing works
“In non-pharmacological trials, a third of the claims of treat-ment superiority based on unpublished scales would not have been made if a published scale had been used [109].”
BackgroundA methodological review of resilience measurement scales
Windle, G., Bennett, K. M., & Noyes, J. (2011) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-9-8 · Cited by 2,079 · see citing works
“[13] found that clinical trials evaluating interventions for people with schizophrenia were almost 40% more likely to report that treatment was effective when they used unpublished scales as opposed to validated measures.”
BackgroundMoher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.004 · Cited by 1,858 · see citing works
Huhn, M., Nikolakopoulou, A., Schneider-Thoma, J., Krause, M., Samara, M., Peter, N., Arndt, T., Bäckers, L., Rothe, P., Cipriani, A., et al. (2019) · The Lancet · doi:10.1016/s0140-6736(19)31135-3 · Cited by 1,753 · see citing works
Better Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement
Ioannidis, J. P. A., Evans, S. J. W., Gøtzsche, P. C., O'Neill, R. T., Altman, D. G., Schulz, K., Moher, D., & for the CONSORT Group* (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-141-10-200411160-00009 · Cited by 1,593 · see citing works
Association Between Duration of Untreated Psychosis and Outcome in Cohorts of First-Episode Patients
Marshall, M., Lewis, S. W., Lockwood, A., Drake, R., Jones, P. B., & Croudace, T. J. (2005) · Archives of General Psychiatry · doi:10.1001/archpsyc.62.9.975 · Cited by 1,595 · see citing works
“Data on primary or secondary outcomes were excluded if they were collected using unpublished scales.(13) Data were extracted independently by 2 raters (M.”
Self-citationReporting of Patient-Reported Outcomes in Randomized Trials: The CONSORT PRO Extension
Calvert, M., Blazeby, J., Altman, D. G., Revicki, D. A., Moher, D., Brundage, M. D., & CONSORT PRO Group, F. T. (2013) · JAMA · doi:10.1001/jama.2013.879 · Cited by 1,386 · see citing works
“If an RCT uses a PRO instrument with psychometric properties that have not yet been published (eg, a new instrument developed for the trial), the authors should provide information on item content of the instrument and evidence regarding its reliability and validity, in an appendix if the article does not allow for su…”
MethodWykes, T., Steel, C., Everitt, B., & Tarrier, N. (2007) · Schizophrenia Bulletin · doi:10.1093/schbul/sbm114 · Cited by 1,029 · see citing works
Systematic review of treatments for atopic eczema.
Hoare, Po, A. L. W., & Williams (2000) · Health Technology Assessment · doi:10.3310/hta4370 · Cited by 789 · see citing works
Family intervention for schizophrenia
Pharoah, F., Mari, J., Rathbone, J., & Wong, W. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000088.pub2 · Cited by 737 · see citing works
Psychoeducation for schizophrenia
Xia, J., Merinder, L. B., & Belgamwar, M. R. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002831.pub2 · Cited by 634 · see citing works
Leucht, S., Leucht, C., Huhn, M., Chaimani, A., Mavridis, D., Helfer, B., Samara, M., Rabaioli, M., Bächer, S., Cipriani, A., et al. (2017) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2017.16121358 · Cited by 542 · see citing works
Helping people with severe mental illness to obtain work: systematic review
Crowther, R. (2001) · BMJ · doi:10.1136/bmj.322.7280.204 · Cited by 538 · see citing works
Early intervention for psychosis
Marshall, M., & Rathbone, J. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004718.pub3 · Cited by 531 · see citing works
Self-citationThorborg, K., Hölmich, P., Christensen, R., Petersen, J., & Roos, E. M. (2011) · British Journal of Sports Medicine · doi:10.1136/bjsm.2010.080937 · Cited by 520 · see citing works
Intensive case management for severe mental illness
Dieterich, M., Irving, C. B., Park, B., & Marshall, M. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007906.pub2 · Cited by 488 · see citing works
Self-citationStandardising outcomes for clinical trials and systematic reviews
Clarke, M. (2007) · Trials · doi:10.1186/1745-6215-8-39 · Cited by 483 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2019) · CHANCE · doi:10.1080/09332480.2019.1579573 · Cited by 473 · see citing works
Why clinical trial outcomes fail to translate into benefits for patients
Heneghan, C., Goldacre, B., & Mahtani, K. R. (2017) · Trials · doi:10.1186/s13063-017-1870-2 · Cited by 404 · see citing works
Markon, K. E., Chmielewski, M., & Miller, C. J. (2011) · Psychological Bulletin · doi:10.1037/a0023678 · Cited by 656 · see citing works
Family intervention for schizophrenia
Pharoah, F., Mari, J. D. J., Rathbone, J., & Wong, W. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000088.pub3 · Cited by 405 · see citing works
Cognitive-Behavioral Interventions to Reduce Suicide Behavior: A Systematic Review and Meta-Analysis
Tarrier, N., Taylor, K., & Gooding, P. (2008) · Behavior Modification · doi:10.1177/0145445507304728 · Cited by 384 · see citing works
Xia (2009) · doi:10.1192/pb.bp.108.021949 · Cited by 381 · see citing works
Self-citationFactors That Can Affect the External Validity of Randomised Controlled Trials
Rothwell, P. M. (2006) · PLoS Clinical Trials · doi:10.1371/journal.pctr.0010009 · Cited by 379 · see citing works
“More worryingly, a review of 2,000 RCTs in schizophrenia identified 640 scales— many of which were devised for the particular RCT and had no supporting data on validity or reliability, but which were more likely to show statistically significant treatment effects than established scales [15].”
BackgroundInvestigating quality of life and health-related quality of life in infertility: a systematic review
Chachamovich, J. R., Chachamovich, E., Ezer, H., Fleck, M. P., Knauth, D., & Passos, E. P. (2010) · Journal of Psychosomatic Obstetrics & Gynecology · doi:10.3109/0167482x.2010.481337 · Cited by 372 · see citing works
“Furthermore, results based on non-validated scales are more prone to demonstrate medication efficacy [31], and inconsistent instruments are associated to biased outcome measurements in clinical trials [32].”
BackgroundElectroconvulsive therapy for schizophrenia
Tharyan, P., & Adams, C. E. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000076.pub2 · Cited by 368 · see citing works
Self-citationOlanzapine versus other atypical antipsychotics for schizophrenia
Komossa, K., Rummel‐Kluge, C., Hunger, H., Schmid, F., Schwarz, S., Duggan, L., Kissling, W., & Leucht, S. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006654.pub2 · Cited by 366 · see citing works
“Unpublished scales are known to be subject to bias in trials of treatments for schizophrenia ( Marshall 2000).”
BackgroundInvolving users in the delivery and evaluation of mental health services: systematic review
Simpson, E. L., & House, A. O. (2002) · BMJ · doi:10.1136/bmj.325.7375.1265 · Cited by 350 · see citing works
Music therapy for people with schizophrenia and schizophrenia-like disorders
Geretsegger, M., Mössler, K. A., Bieleninik, Ł., Chen, X., Heldal, T. O., & Gold, C. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004025.pub4 · Cited by 357 · see citing works
Clozapine versus typical neuroleptic medication for schizophrenia
Essali, A., Hasan, N. A., Li, C., & Rathbone, J. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000059.pub2 · Cited by 326 · see citing works
Compulsory community and involuntary outpatient treatment for people with severe mental disorders
Kisely, S. R., & Campbell, L. A. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004408.pub4 · Cited by 320 · see citing works
Crisis intervention for people with severe mental illnesses
Murphy, S., Irving, C. B., Adams, C. E., & Waqar, M. U. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001087.pub5 · Cited by 321 · see citing works
Self-citationPsychosocial interventions for self-harm in adults
Hawton, K. E., Witt, K., Salisbury, T. T., Arensman, E., Gunnell, D., Hazell, P., Townsend, E., & van Heeringen, K. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012189 · Cited by 314 · see citing works
Tarrier, N., & Wykes, T. (2004) · Behaviour Research and Therapy · doi:10.1016/j.brat.2004.06.020 · Cited by 301 · see citing works
Zimmermann, G., Favrod, J., Trieu, V. H., & Pomini, V. (2005) · Schizophrenia Research · doi:10.1016/j.schres.2005.02.018 · Cited by 371 · see citing works
“Since Marshall et al. (2000) suggested that the use of an unpublished scale may be a source of bias in schizophrenia trials, we only included data from validated scales (e.”
BackgroundVocational rehabilitation for people with severe mental illness
Crowther, R., Marshall, M., Bond, G. R., & Huxley, P. J. (2001) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003080 · Cited by 293 · see citing works
Self-citationAripiprazole for schizophrenia
El-Sayeh, H. G. G., & Morganti, C. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004578.pub3 · Cited by 291 · see citing works
Maintenance treatment with antipsychotic drugs for schizophrenia
Leucht, S., Tardy, M., Komossa, K., Heres, S., Kissling, W., & Davis, J. M. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008016.pub2 · Cited by 287 · see citing works
Psychosocial Interventions for People With Both Severe Mental Illness and Substance Misuse
Hunt, G. E., Siegfried, N., Morley, K. C., Sitharthan, T., & Cleary, M. L. (2013) · Schizophrenia Bulletin · doi:10.1093/schbul/sbt160 · Cited by 282 · see citing works
Gooding, P., & Tarrier, N. (2009) · Behaviour Research and Therapy · doi:10.1016/j.brat.2009.04.002 · Cited by 267 · see citing works
“…& Eggar, 2001; Juni, Witschi, Bloch, & Eggar, 1999; Kazdin & Bass, 5 For in depth information on any of the analyses conducted, and the calculations P. Gooding, N. Tarrier / Behaviour Research and Therapy 47 (2009) 592–607594 1989; Marshall et al., 2000; Moher et al., 1998; Sterne et al., 2002).”
BackgroundCognitive behaviour therapy for schizophrenia
Jones, C., Cormac, I., Silveira da Mota Neto, J. I., & Campbell, C. (2004) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000524.pub2 · Cited by 269 · see citing works
RetractedInterventions for self-harm in children and adolescents
Hawton, K., Witt, K. G., Taylor Salisbury, T. L., Arensman, E., Gunnell, D., Townsend, E., van Heeringen, K., & Hazell, P. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012013 · Cited by 261 · see citing works
Intensive case management for severe mental illness
Dieterich, M., Irving, C. B., Bergman, H., Khokhar, M. A., Park, B., & Marshall, M. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007906.pub3 · Cited by 271 · see citing works
Self-citationQuetiapine versus other atypical antipsychotics for schizophrenia
Komossa, K., Rummel‐Kluge, C., Schmid, F., Hunger, H., Schwarz, S., Srisurapanont, M., Kissling, W., & Leucht, S. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006625.pub2 · Cited by 261 · see citing works
Haloperidol versus placebo for schizophrenia
Irving, C. B., Adams, C. E., & Lawrie, S. M. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003082.pub2 · Cited by 247 · see citing works
Self-citationDuggan, L., Fenton, M., Rathbone, J., Dardennes, R. M., El-Dosoky, A., & Indran, S. K. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001359.pub2 · Cited by 242 · see citing works
Self-citationZiprasidone versus other atypical antipsychotics for schizophrenia
Komossa, K., Rummel‐Kluge, C., Hunger, H., Schwarz, S., Bhoopathi, P. S., Kissling, W., & Leucht, S. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006627.pub2 · Cited by 241 · see citing works
“Unpublished scales are known to be subject to bias in trials of treatments for schizophrenia ( Marshall 2000).”
BackgroundChlorpromazine versus placebo for schizophrenia
Adams, C. E., Awad, G. A., Rathbone, J., & Thornley, B. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000284.pub2 · Cited by 241 · see citing works
Self-citationMalone, D., Marriott, S. V. L., Newton‐Howes, G., Simmonds, S., & Tyrer, P. J. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000270.pub2 · Cited by 228 · see citing works
Interventions to reduce weight gain in schizophrenia
Faulkner, G., Cohn, T. A., & Remington, G. J. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005148.pub2 · Cited by 226 · see citing works
Chakrabarti, A., Bagnall, A., Chue, P. S., Fenton, M., Palanisamy, V., Wong, W., & Xia, J. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001943.pub2 · Cited by 222 · see citing works
Self-citationHaloperidol plus promethazine for psychosis-induced aggression
Huf, G., Alexander, J., Allen, M. H., & Raveendran, N. S. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005146.pub2 · Cited by 223 · see citing works
Patient reported outcomes in orthopaedics
Gagnier, J. J. (2017) · Journal of Orthopaedic Research® · doi:10.1002/jor.23604 · Cited by 220 · see citing works
Validation therapy for dementia
Neal, M., & Wright, P. B. (2003) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001394 · Cited by 219 · see citing works
Paliperidone Palmitate for Schizophrenia
Nussbaum, A. M., & Stroup, T. S. (2012) · Schizophrenia Bulletin · doi:10.1093/schbul/sbs099 · Cited by 219 · see citing works
Thioridazine for schizophrenia
Fenton, M., Rathbone, J., & Reilly, J. G. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001944.pub2 · Cited by 210 · see citing works
Self-citationSulpiride augmentation for schizophrenia
Wang, J., Omori, I. M., Fenton, M., & Soares, B. G. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008125.pub2 · Cited by 206 · see citing works
Self-citationHaloperidol versus chlorpromazine for schizophrenia
Leucht, C., Kitzmantel, M., Kane, J. M., Leucht, S., & Chua, W. L. L. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004278.pub2 · Cited by 206 · see citing works
Amisulpride versus other atypical antipsychotics for schizophrenia
Komossa, K., Rummel‐Kluge, C., Hunger, H., Schmid, F., Schwarz, S., Neto, J. I. S. D. M., Kissling, W., & Leucht, S. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006624.pub2 · Cited by 205 · see citing works
Outcome Measures of Disease Severity in Atopic Eczema
Charman, C. R., & Williams, H. C. (2000) · Archives of Dermatology · doi:10.1001/archderm.136.6.763 · Cited by 202 · see citing works
Which Reported Estimate of the Prevalence of Malnutrition After Stroke Is Valid?
Foley, N. C., Salter, K. L., Robertson, J. C., Teasell, R. W., & Woodbury, M. G. (2009) · Stroke · doi:10.1161/strokeaha.108.518910 · Cited by 200 · see citing works
Psychology's Replication Crisis and Clinical Psychological Science
Tackett, J. L., Brandes, C. M., King, K. M., & Markon, K. E. (2019) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050718-095710 · Cited by 196 · see citing works
“The possibility of this phenomenon gives weight to the importance of developing and using well-validated and justified measures (Marshall et al. 2000).”
BackgroundGreenhalgh, J., Knight, C., Hind, D., Beverley, C., & Walters, S. J. (2005) · Health Technology Assessment · doi:10.3310/hta9090 · Cited by 197 · see citing works
Oral paliperidone for schizophrenia
Nussbaum, A. M., & Stroup, T. S. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006369.pub2 · Cited by 195 · see citing works
Depot perphenazine decanoate and enanthate for schizophrenia
David, A. S., Quraishi, S. N., & Rathbone, J. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001717.pub2 · Cited by 196 · see citing works
Mokkink, L. B., Terwee, C. B., Stratford, P. W., Alonso, J., Patrick, D. L., Riphagen, I., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2009) · Quality of Life Research · doi:10.1007/s11136-009-9451-9 · Cited by 186 · see citing works
“[1] showed that in schizophrenia trials authors were more likely to report that treatment was superior to control when an unpublished instrument was used in the comparison, rather than a published instrument.”
Combined psychotherapy plus antidepressants for panic disorder with or without agoraphobia
Furukawa, T. A., Watanabe, N., & Churchill, R. C. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004364.pub2 · Cited by 195 · see citing works
Quetiapine versus typical antipsychotic medications for schizophrenia
Suttajit, S., Srisurapanont, M., Xia, J., Suttajit, S., Maneeton, B., & Maneeton, N. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007815.pub2 · Cited by 192 · see citing works
Pimozide for schizophrenia or related psychoses
Rathbone, J., & McMonagle, T. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001949.pub2 · Cited by 191 · see citing works
Wang, Y., Xia, J., Helfer, B., Li, C., & Leucht, S. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004028.pub4 · Cited by 188 · see citing works
Molindone for schizophrenia and severe mental illness
Bagnall, A., Fenton, M., Kleijnen, J., & Lewis, R. A. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002083.pub2 · Cited by 188 · see citing works
Self-citationOutcome selection bias in meta-analysis
Williamson, P. R., Gamble, C., Altman, D. G., & Hutton, J. L. (2005) · Statistical Methods in Medical Research · doi:10.1191/0962280205sm415oa · Cited by 188 · see citing works
Supported employment for adults with severe mental illness
Kinoshita, Y., Furukawa, T. A., Omori, I. M., Watanabe, N., Marshall, M., Bond, G. R., Huxley, P. J., & Kingdon, D. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008297 · Cited by 188 · see citing works
Self-citationClozapine versus other atypical antipsychotics for schizophrenia
Lobos, C. A., Komossa, K., Rummel‐Kluge, C., Hunger, H., Schmid, F., Schwarz, S., & Leucht, S. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006633.pub2 · Cited by 187 · see citing works
Chlorpromazine dose for people with schizophrenia
Liu, X., & Haan, S. D. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007778 · Cited by 187 · see citing works
Supported employment for adults with severe mental illness
Kinoshita, Y., Furukawa, T. A., Kinoshita, K., Honyashiki, M., Omori, I. M., Marshall, M., Bond, G. R., Huxley, P., Amano, N., & Kingdon, D. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008297.pub2 · Cited by 182 · see citing works
Self-citationSertindole versus other atypical antipsychotics for schizophrenia
Komossa, K., Rummel‐Kluge, C., Hunger, H., Schwarz, S., Schmid, F., Lewis, R. A., Kissling, W., & Leucht, S. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006752.pub2 · Cited by 182 · see citing works
Music therapy for people with schizophrenia and schizophrenia-like disorders
Mössler, K. A., Chen, X., Heldal, T. O., & Gold, C. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004025.pub3 · Cited by 188 · see citing works
Haloperidol for psychosis-induced aggression or agitation (rapid tranquillisation)
Ostinelli, E. G., Powney, M. J., Li, X., & Adams, C. E. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009377.pub3 · Cited by 179 · see citing works
Self-citationSulpiride versus placebo for schizophrenia
Omori, I. M., & Wang, J. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007811 · Cited by 173 · see citing works
Leucht, S., Helfer, B., Dold, M., Kissling, W., & McGrath, J. J. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003834.pub3 · Cited by 173 · see citing works
Cognitive Behavior Therapy Versus Other Psychosocial Treatments for Schizophrenia
Jones, C., Hacker, D., Cormac, I., Meaden, A., & Irving, C. B. (2012) · Schizophrenia Bulletin · doi:10.1093/schbul/sbs090 · Cited by 172 · see citing works
Maintenance treatment with antipsychotic drugs for schizophrenia
Ceraso, A., Lin, J., Schneider‐Thoma, J., Siafis, S., Tardy, M., Komossa, K., Heres, S., Kissling, W., Davis, J. M., & Leucht, S. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008016.pub3 · Cited by 176 · see citing works
Intercessory prayer for the alleviation of ill health
Roberts, L., Ahmed, I., & Davison, A. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000368.pub3 · Cited by 173 · see citing works
Chlorpromazine versus placebo for schizophrenia
Adams, C. E., Awad, G. A., Rathbone, J., Thornley, B., & Soares‐Weiser, K. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000284.pub3 · Cited by 167 · see citing works
Self-citationBenzodiazepines for schizophrenia
Dold, M., Li, C., Tardy, M., Khorsand, V., Gillies, D. M., & Leucht, S. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006391.pub2 · Cited by 169 · see citing works
Psychotherapy plus antidepressant for panic disorder with or without agoraphobia
Furukawa, T. A., Watanabe, N., & Churchill, R. C. (2006) · The British Journal of Psychiatry · doi:10.1192/bjp.188.4.305 · Cited by 165 · see citing works
“One study indicated that rating scales which have not been validated or standardised are more likely to report statistically significant findings (Marshall et al, 2000).”
BackgroundMarshall, M., Sledge, W., Wiersma, D., Crowther, R., Kluiter, H., Bond, G. R., Almaraz-Serrano, A., Roberts, C., Huxley, P., Creed, F., et al. (2001) · Health Technology Assessment · doi:10.3310/hta5210 · Cited by 158 · see citing works
“Age (years) ≤24 2 (4) 4 (10) 14 (17) 14 (17) 15 (15) 15 (16) 25–34 20 (43) 10 (25) 23 (27) 24 (29) 43 (44) 42 (46) 35–44 7 (15) 6 (15) 25 (30) 17 (20) 23 (23) 20 (22) 45–54 8 (17) 7 (18) 13 (15) 9 (11) 10 (10) 11 (12) ≥55 10 (21) 13 (32) 9 (11) 20 (24) 7 (7) 3 (3)”
Self-citationCharman, C. R., Chambers, C., & Williams, H. C. (2003) · Journal of Investigative Dermatology · doi:10.1046/j.1523-1747.2003.12251.x · Cited by 159 · see citing works
Haloperidol versus placebo for schizophrenia
Adams, C. E., Bergman, H., Irving, C. B., & Lawrie, S. M. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003082.pub3 · Cited by 157 · see citing works
Self-citationCognitive behaviour therapy for primary insomnia: Can we rest yet?
Harvey, A. G., & Tang, N. K. Y. (2003) · Sleep Medicine Reviews · doi:10.1053/smrv.2002.0266 · Cited by 158 · see citing works
CONSORT extension for reporting N-of-1 trials (CENT) 2015: Explanation and elaboration
Shamseer, L., Sampson, M., Bukutu, C., Schmid, C. H., Nikles, J., Tate, R. L., Johnston, B. C., Zucker, D. R., Shadish, W. R., Kravitz, R. L., et al. (2015) · BMJ · doi:10.1136/bmj.h1793 · Cited by 154 · see citing works
- Authors
- Marshall, M., Lockwood, A., Bradley, C., Adams, C., Joy, C., & Fenton, M.
- Affiliations
- University of Manchester (GB)
- University of Oxford (GB)
- Cochrane (GB)
- Corresponding author
- Marshall, Max
- Venue
- British Journal of Psychiatry
- Publisher
- Royal College of Psychiatrists
- Published
- 2000-03
- Volume
- 176
- Issue
- 3
- Pages
- 249-252
- ISSN
- 0007-1250 (ISSN-L)
- 1472-1465
- OpenAlex
- W2023839473
- Semantic Scholar
- b21cfd8f6a2550339df128b4a2661f9cdfc0d521
- Cited by
- 870
- References
- 6
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 23 · 2023: 12 · 2022: 16 · 2021: 32 · 2020: 20 · 2019: 32 · 2018: 39 · 2017: 48 · 2016: 47 · 2015: 58 · 2014: 51 · 2013: 53 · 2012: 58
- FWCI
- 41.35
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- The British Journal of Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Psychiatry and Mental health › Schizophrenia research and treatment
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- publication bias, schizophrenia, randomized controlled trials, treatment superiority, outcome measures, non-pharmacological treatment
- MeSH terms
- Humans, Peer Review, Psychiatric Status Rating Scales, Publishing, Schizophrenia, Reproducibility of Results, Observer Variation, Randomized Controlled Trials as Topic, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
