CoChat
Sign Up
Cited by 61Meta-analysis and systematic reviewsPharmaceutical industry and healthcarePsychology Research and Bibliometrics

Quality of randomized controlled trials of new generation antidepressants and antipsychotics identified in the China National Knowledge Infrastructure (CNKI): a literature and telephone interview study

Open access
Tong, Zheng, Li, Fangzhou, Ogawa, Yusuke, Watanabe, Norio, Furukawa, Toshi A.·2018·BMC Medical Research Methodology
DOI 10.1186/s12874-018-0554-2PMC PMC6154421
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: We are witnessing an exponential increase in the number of randomized controlled trials (RCTs) reported from mainland China. The increase is particularly notable in the field of new generation antidepressants and antipsychotics. Several previous studies have raised doubts regarding their quality. However, the quality of most recent RCTs published in China may have improved. METHODS: We searched RCTs that examined new generation antidepressants and antipsychotics published between 2013 and 2016 in the China National Knowledge Infrastructure (CNKI), the largest database of scientific publications in China. We interviewed the authors of a random subset of the identified references. We assessed the methodological rigor of each study based on the published reports and telephone interviews with the authors using six methodological domains adapted from the Cochrane's risk of bias tool. RESULTS: The final sample consisted of 138 studies, for which we interviewed 58 authors; the authors of 51 studies declined the interview, and the authors of 29 studies could not be contacted. The 51 studies with refused interviews were significantly less likely to be reported from university-affiliated hospitals and were less likely to be published in Chinese core journals. Based on the published reports, most of the 58 studies were assessed to be at unclear risk of bias in most methodological domains. After the interview, only 10 studies were assessed to be at low risk of bias for sequence generation and allocation concealment. Assuming that the studies for which the authors declined interviews had an unclear risk, the proportion of RCTs at low risk of bias in both sequence generation and allocation concealment was 9.2% (10/109, 95% confidence interval [CI]: 5.0 to 16.2). The interviews indicated that the studies were at high risk of bias for most of the other domains. CONCLUSION: In general, RCTs that evaluate new generation antidepressants or antipsychotics and are indexed in the CNKI continue to be of low quality. When conducting systematic reviews and meta-analyses in this field, it would be wise to include a specialist from China as a coresearcher to help assess the risk of bias in the identified studies.

AI snapshot

Written from the paper’s full text.

TL;DR

In a literature and telephone interview study of RCTs of new-generation antidepressants and antipsychotics indexed in CNKI (2013–2016), most published reports were at unclear risk of bias and interviews often revealed high risk. Only 9.2% (10/109) were estimated to be at low risk of bias for both sequence generation and allocation concealment when interview refusals were assumed unclear.

Key findings

  • Of 138 included studies, 58 authors were interviewed, 51 refused, and 29 could not be contacted; refused-interview reports were significantly less likely to be from university-affiliated hospitals and less likely to be published in core journals (e.g., CSTPCD core journals: 9.8% vs 36.2%, p < 0.05).
  • Based on published reports, most of the 58 interviewed studies were at unclear risk of bias; no study was rated low risk for allocation concealment or registered protocol, 16/58 (27.6%) were low risk for sequence generation, 2/58 (3.4%) for blinding of at least one object, and 52/58 (89.7%) for intention-to-treat analysis.
  • After interviews, only 10/58 (17.2%, 95% CI 9.6–29.0) were low risk for both sequence generation and allocation concealment; assuming refused interviews were unclear, this was 9.2% (10/109, 95% CI 5.0–16.2).
  • Interviews indicated high risk for most other domains: 37 authors admitted not intending to blind, 30 reports were high risk for intention-to-treat analysis, 52 authors admitted no trial registration, no author provided a protocol, and some authors admitted ghostwriting, fabricated data, or reporting retrospective studies as RCTs.

Methods

  • Searched CNKI on March 8, 2017 for Chinese-language RCTs of new-generation antidepressants or antipsychotics published 2013–2016; included 138 studies and completed semi-structured telephone interviews with 58 authors.
  • Assessed risk of bias in six domains adapted from the Cochrane tool (sequence generation, allocation concealment, blinding, intention-to-treat analysis, protocol registration, provision of protocol), first from published reports and then after interviews; two native Chinese researchers independently assessed.
  • Compared characteristics of interviewed, refused, and uncontactable reports; primary outcome was proportion at low risk for both sequence generation and allocation concealment, with sensitivity analysis assuming refusals unclear or high risk; used JMP software.

Limitations

  • Stated: findings are limited to RCTs of new-generation antidepressants and antipsychotics in CNKI and may not generalize to other specialties or databases; CNKI lacks a recommended-journal index, and results may differ if limited to core journals or PubMed.
  • Stated: authors’ answers were often ambiguous or vague, interviews were short and conducted on busy hospital lines, no central randomization scheme was reported, and the response rate was 42.0% (58/138) with a 37.0% refusal rate versus 2.8% in the previous study; refused reports appeared lower quality.
  • Evident: the study relied on self-report and could not verify protocols or source data; no author provided a protocol, and some authors admitted fabrication or ghostwriting, so interview-based risk-of-bias judgments may themselves be unreliable.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12874-018-0554-2

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 47 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Among the included studies, the 51 reports with refused interviews were significantly less likely to be conducted in university-affiliated hospitals than reports with successful interviews.

    “The 51 studies with refused interviews were significantly less likely to be reported from university-affiliated hospitals and were less likely to be published in Chinese core journals.”
    • Also stated in: § Characteristics of reports for which authors could not be contacted or refused the interview · plain
      “The reports with refused interviews were significantly less likely to be conducted in university-affiliated hospitals and were less likely to be published in core journals of the China Scientific and Technical Papers and Citations Database than the reports with authors who agreed to be interviewed.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Authors of five reports claimed they used outcome assessors from other departments to maintain blinding of outcome assessors.

    “The authors of five reports claimed they used outcome assessors from other departments to maintain the blinding of outcome assessors.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Authors of nine reports misunderstood blinding as "not telling them that they were in a study" or "not telling the patients which pills they were taking".

    “Moreover, the authors of nine reports misunderstood blinding for “not telling them that they were in a study” or “not telling the patients which pills they were taking”.”
    Quote in the paperSubstantial claim
  • Findingown data

    One report was at low risk for blinding only participants by reporting the same appearance and taste of pills.

    “One report was at low rsik for blinding only participants by reporting the same appearance and taste of pills.”
    Quote in the paperNot a substantial claim
  • Findingown data

    During the interview, some authors did not completely understand the concept of random allocation because they continued to talk about inclusion criteria instead of the randomization method when asked about sequence generation.

    “During the interview, we determined some authors did not completely understand the concept of random allocation because they continued to talk about the inclusion criteria instead of the randomization method when asked about sequence generation.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 40
  • The quote only partly states this claim. 1
  • Classified as probably not a substantial claim. 24

  • May combine two claims.

    Findingown data

    After the interview, among the 32 reports assessed to be at unclear risk for sequence generation based on the literature, 20 exhibited high risk of bias, using alternation depending on time order (n=11), following clinicians' prescription (n=6), or other inappropriate methods (n=3).

    “After the interview, we found that among the 32 reports assessed to be at unclear rsik based on the literature, 20 reports exhibited a high risk of bias, using alternation depending on the time order (n = 11), following the clinicians’ prescription (n = 6) or other inappropriate methods (n = 3).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Among the 16 reports originally assessed to be at low risk of bias for sequence generation, after interview eight were assessed to be at high risk, one at unclear risk, and only seven at low risk.

    “Among the 16 reports originally assessed to be at low risk of bias, eight reports were assessed to be at high rsik, one report was assessed to be unclear, and only 7 reports were assessed to be at low rsik.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Altogether, 30 reports were assessed to be at high risk of bias based on interview results, with 21 authors indicating that drop-outs were excluded from analysis and reporting.

    “Altogether, 30 reports were assessed to be at high risk of bias based on the interview results, with 21 authors indicating that drop-outs were excluded from analysis and reporting.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    After the interview, an additional 36 reports were assessed to be at high risk of bias for allocation concealment, and 11 reports were assessed to be at low risk of bias.

    “After the interview, an additional 36 reports were assessed to be at high risk of bias, and only 11 reports were assessed to be at low risk of bias, as the authors claimed that the treatment assignment was managed by personnel separate and independent from the doctors who recruited patients and that the doctors who enrolled the next patient were unaware of the next allocation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Ten reports were assessed to be at high risk of bias for allocation concealment because they explicitly indicated that they used alternation depending on the time order, and their studies were all reported to be conducted in one department of a single hospital.

    “Ten reports were assessed to be at high risk of bias because they explicitly indicated that they used alternation depending on the time order, and their studies were all reported to be conducted in one department of a single hospital.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Three reports failed to report the number of participants for the outcome and were assessed to be at unclear risk of bias.

    “Three reports failed to report the number of participants for the outcome and were assessed to be at unclear risk of bias.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingreasoning

    These 56 reports were assessed to be at unclear risk of bias for blinding.

    “These 56 reports were assessed to be at unclear risk of bias.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The study finally included 138 studies to reach the pre-specified sample size and completed 58 interviews.

    “We finally included 138 studies to reach the pre-specified sample size and completed 58 interviews.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Fifty-one reports were assessed at low risk of bias for intention-to-treat analysis, with 40 indicating no drop-out at all.

    “Fifty-one reports were assessed to be at low risk of bias with regard to the intention-to-treat analysis, with 40 reports indicating no drop-out at all.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Fewer than one in five studies reported ethical approval, and less than half reported obtaining informed consent from participants.

    “Fewer than one in five studies reported ethical approval, and less than half reported obtaining informed consent from the participants.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Authors of 52 reports admitted to no registration, and for the remaining six reports the authors' answers were ambiguous.

    “The authors of 52 reports admitted to no registration, and for the remaining six reports, the authors’ answers were ambiguous.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The published reports often lacked methodological information and were mostly assessed to be at unclear risk of bias.

    “The published reports often lacked methodological information and were mostly assessed to be at unclear risk of bias.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The included studies were mostly small with sample sizes of less than 60 per arm and reported extremely low drop-out rates.

    “The included studies were mostly small with sample sizes of less than 60 per arm and reported extremely low drop-out rates.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    The response rate was 42.0% (58 of 138), substantially lower than the previous study's 73.1%.

    “Third, the response rate was 42.0% (58 of 138), which is substantially lower than that of the previous study, 73.1% [9].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9] Wu T, Li Y, Bian Z, Liu G, Moher D. Randomized trials published in some Chinese journals: how many are randomized? Trials. 2009;10:46. source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Authors of 13 reports indicated they did not quite remember the drop-outs; their reports were assessed as unclear.

    “The authors of 13 reports indicated that they did not quite remember the drop-outs; thus, their reports were assessed as unclear.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Some authors admitted to falsifying their studies by reporting retrospective data as RCTs or fabricating their studies from data in the literature or using a ghostwriting service.

    “Moreover, we found that some authors admitted to falsifying their studies by reporting their retrospective data as RCTs or fabricating their studies from data in the literature or using a ghostwriting service.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    After the interview, many studies originally found to be at unclear risk and some studies at low risk were judged to be at high risk of bias.

    “After the interview, many of the studies originally found to be at unclear risk and some studies at low risk were judged to be at high risk of bias.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Telephone interview responses accurately reflect the actual conduct of the trials.

  • May combine two claims.

    Findingown data

    As far as the investigators could ascertain, none of the studies in the sample was registered in a Chinese or international trial registry or was indexed in PubMed.

    “As far as we could ascertain, none of the studies in our sample was registered in a Chinese or international trial registry or was indexed in PubMed.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In 15 reports assessed at low risk of bias, the reported drop-out rate was very low, and authors provided reasonable explanations such as all participants being inpatients, a short observational period, or benign side-effects of the drugs.

    “In the 15 reports assessed to be at low risk of bias, the drop-out rate reported in the paper was very low; however, the authors provided reasonable explanations, for example, that the participants included in the study were all inpatients, the observational period was short, or the side-effects of the drugs were benign.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Authors of 36 reports admitted they had no protocol or only simple plans; these studies were rated at high risk of bias for protocol.

    “The authors of 36 reports admitted that they had no protocol or had only simple plans: these studies were rated to be at high risk of bias for protocol.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The lack of integration of Chinese trials into the world medical literature may be due in part to the language barrier and is also likely due to the purported low methodological quality of most studies published in China.

    “This lack may be due, in part, to the language barrier; however, it is also likely due to the purported low methodological quality of most studies published in China.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    More than ten years have passed since Wu et al.'s study, and the quality of recent RCTs published in China may have improved in the interim.

    “More than ten years have passed since Wu et al.’s epoch-making study [9], and the quality of recent RCTs published in China may have improved in the interim.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [9] Wu T, Li Y, Bian Z, Liu G, Moher D. Randomized trials published in some Chinese journals: how many are randomized? Trials. 2009;10:46. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Wu et al. reported that only 6.8% of published RCTs between 1995 and 2005 identified in the CNKI adhered to accepted methodology for randomization.

    “First, Wu et al. [9] reported after interviewing 2235 authors by telephone that only 6.8% of published randomized controlled trials (RCTs) between 1995 and 2005 identified in the China National Knowledge Infrastructure (CNKI) adhered to accepted methodology for randomization.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9] Wu T, Li Y, Bian Z, Liu G, Moher D. Randomized trials published in some Chinese journals: how many are randomized? Trials. 2009;10:46. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Clinical studies from China have increased in recent years in both absolute quantity and relative share in the world.

    “Clinical studies from China have been attracting worldwide attention as their publications have increased in recent years in both absolute quantity and relative share in the world [1–3].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [1] Pu QH, Lyu QJ, Su HY. Bibliometric analysis of scientific publications in transplantation journals from mainland China, Japan, South Korea and Taiwan between 2006 and 2015. BMJ Open. 2016;6(8):e011623. source

      Not checked in the free run.

    • cites [2] Chen K, Yao Q, Sun J, He ZF, Yao L, Liu ZY. International publication trends and collaboration performance of China in healthcare science and services research. Isr J Health Policy Res. 2016;5(1). source

      Not checked in the free run.

    • cites [3] Zou Y, Li Q, Xu W. Scientific research output in orthopaedics from China and other top-ranking countries: a 10-year survey of the literature. BMJ Open. 2016;6(9):e011605. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Two researchers who are native speakers of Chinese independently completed all risk of bias assessments.

    “Two researchers who are native speakers of Chinese independently completed all assessments.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    News in the past few years has implicated an alarmingly high proportion of plagiarism, falsification or fabrication in the scientific and clinical literature from China.

    “Moreover, news in the past few years has implicated an alarmingly high proportion of plagiarism, falsification or fabrication in the scientific and clinical literature from China [13–15].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [13] Woodhead M. 80% of China's clinical trial data are fraudulent, investigation finds. BMJ. 2016;355:i5396. source

      Not checked in the free run.

    • cites [14] Qiu J. Publish or perish in China. Nature. 2010;463(7278):142–3. source

      Not checked in the free run.

    • cites [15] Normile D. China cracks down on fraud. Science. 2017;357(6350):435. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Wu et al. found that 85.6% of Chinese authors did not completely understand the principles of randomization when they claimed that their trials were RCTs.

    “They found that 85.6% of Chinese authors did not completely understand the principles of randomization when they claimed that their trials were RCTs [9].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9] Wu T, Li Y, Bian Z, Liu G, Moher D. Randomized trials published in some Chinese journals: how many are randomized? Trials. 2009;10:46. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The 2014 update of the Cochrane review of aripiprazole for schizophrenia identified 170 new trials, 162 of which had been conducted in China, in addition to the 4 trials in the original 2009 review.

    “For example, the 2014 update of the Cochrane review of aripiprazole for schizophrenia [5] identified 170 new trials, 162 of which had been conducted in China, in addition to the 4 trials in their original 2009 review [6].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [5] Khanna P, Suo T, Komossa K, Ma H, Rummel-Kluge C, El-Sayeh HG, Leucht S, Xia J. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst Rev. 2014;(1):Cd006569. source

      Not checked in the free run.

    • cites [6] Komossa K, Rummel-Kluge C, Schmid F, Hunger H, Schwarz S, El-Sayeh HG, Kissling W, Leucht S. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst Rev. 2009;4:CD006569.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    All interviews were recorded and independently assessed by two researchers.

    “All interviews were recorded and independently assessed by two researchers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    A sensitivity analysis for the primary outcome was conducted, assuming the reports with refused interviews to be at an unclear or high risk of bias.

    “A sensitivity analysis for the primary outcome was conducted, assuming the reports with refused interviews to be at an unclear or high risk of bias.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The studies for which the authors declined interviews had an unclear risk of bias.

  • May combine two claims.

    Premisecitation

    Many Chinese trials can be accessed only through searches in Chinese databases and have not been adequately integrated into the world medical literature.

    “However, many Chinese trials can be accessed only through searches in the Chinese databases [4] and have not been adequately integrated into the world medical literature.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [4] Zhang Y, Becker T, Ma Y, Koesters M. A systematic review of Chinese randomized clinical trials of SSRI treatment of depression. BMC Psychiatry. 2014;14:245. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    This study was approved by the ethics committee of Kyoto University Graduate School of Medicine (registration number R1019).

    “This study has been approved by the ethics committee of Kyoto University Graduate School of Medicine (registration number R1019).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    Risk of bias was assessed using a tool adapted from the Cochrane risk of bias tool with six domains.

    “The tool for assessing the risk of bias was adapted from the Cochrane’s risk of bias tool [8] and included the following six domains: 1) sequence generation, 2) allocation concealment, 3) blinding of participants, personnel and outcome assessors, and 4) intention-to-treat analysis, in addition to 5) protocol registration and 6) provision of the protocol (only for the telephone interview).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [8] Higgins JP, Green S (eds.): Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011] Available from http://handbook-5-1.cochrane.org/ ; 2011.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    Inadequate reporting of RCTs by Chinese authors makes direct contact with the authors a sine qua non to assess their real quality.

    “Inadequate reporting of RCTs by Chinese authors makes direct contact with the authors a sine qua non to assess their real quality.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Notwithstanding the very low proportion of RCTs at low risk of bias in random allocation identifiable in the CNKI, the sheer volume of clinical trials from mainland China makes them non-ignorable.

    “Notwithstanding the very low proportion of RCTs at low risk of bias in random allocation identifiable in the CNKI, the sheer volume of clinical trials from mainland China makes them non-ignorable.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    The investigators had to expend enormous efforts to contact the authors due first to a lack of corresponding information in the publications and second to some authors' uncooperative attitudes.

    “We had to expend enormous efforts to contact the authors due first to a lack of corresponding information in the publications and second to some authors’ uncooperative attitudes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Although the investigators attempted to assess the actual conduct of the studies, authors' answers were often ambiguous and vague, so these instances were rated as unclear risk of bias.

    “Second, although we attempted to assess the actual conduct of the studies, the authors’ answers were often ambiguous and remained vague, such as “I don’t remember clearly, but I suppose so.”. We had to rate these instances as being at unclear risk of bias.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Of the randomly selected 138 reports, 29 authors could not be contacted, 51 authors were contacted but refused the interview, and 58 authors were successfully interviewed (42.0%).

    “Of the randomly selected 138 reports, 29 authors could not be contacted, and 51 authors were contacted but refused the interview; we were able to successfully conduct semi-structured interviews with 58 authors (42.0%).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The Chinese government is cognizant of the need for and is determined to invest in progress in research integrity in China.

    “It is important news that the Chinese government is also cognizant of the need for and is determined to invest in, progress in research integrity in China [34, 35]: we look forward to seeing positive changes in the near future.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [34] Reforming research in China. Lancet 2007, 369(9565):880. source

      Not checked in the free run.

    • cites [35] Guo L. China boosts medical research. Lancet. 2010;375(9716):711. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    The investigators' experiences indicate that identifying RCTs at low risk of bias among those from the CNKI can be onerous: simple inquiry with study authors may not be sufficient, and ascertaining the pre-published protocol and source data may be required.

    “However, our experiences in the course of the present study indicate that identifying RCTs at low risk of bias among those from the CNKI can be an onerous task: simple inquiry with the study authors may not be sufficient; ascertaining the pre-published protocol and the source data may be required [13].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [13] Woodhead M. 80% of China's clinical trial data are fraudulent, investigation finds. BMJ. 2016;355:i5396. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Other research has indicated similar results and found that Chinese authors had a low understanding of protocols.

    “Other research has indicated similar results and found that Chinese authors had a low understanding of protocols [21].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] Wu T, Bian Z, Shang H, Li Y. Innovation of clinical trials in China: commentary on the publication of “CONSORT extension for Chinese herbal medicine formulas 2017: recommendations, explanation, and elaboration”. J Evid Based Med. 2017;10(3):155–62. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    Assuming 10-20% of trials identified in the CNKI are at low risk for both sequence generation and allocation concealment, 100-300 trials can be identified in the CNKI in recent years.

    “Assuming 10–20% of the trials identified in the CNKI to be at low rsik for both sequence generation and allocation concealment, 100–300 trials can be identified in the CNKI in recent years.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: 10–20% of the trials identified in the CNKI are at low risk for both sequence generation and allocation concealment.

  • May combine two claims.

    Interpretationown data

    The RCTs that examined antidepressants and antipsychotics published in the most recent years in China and identified in the CNKI were not only poor overall in reporting but were also often inadequate in their conduct.

    “In conclusion, the RCTs that examined antidepressants and antipsychotics published in the most recent years in China and identified in the CNKI were not only poor overall in reporting but were also often inadequate in their conduct.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Telephone interview responses accurately reflect the actual conduct of the trials.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    The results would also be different if trials from China published in international databases such as PubMed were assessed, as higher quality studies from China are more likely to be published in English rather than in Chinese.

    “The results would also be different if we assessed trials from China published in international databases, such as PubMed, as higher quality studies from China are more likely to be published in English rather than in Chinese.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The investigators identified differences in quality-related characteristics, which imply that reports with refused interviews have a lower quality than reports with successful interviews.

    “We also identified differences in the quality-related characteristics, which imply that reports with refused interviews have a lower quality than reports with successful interviews.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    In some cases, the investigators had to provide a list of possible answers as hints, and most telephone interviews were conducted on the busy hospital office line within a short time, which may have limited the quality of the interview.

    “In some cases, we also had to provide a list of possible answers as hints, and most of the telephone interviews were conducted on the busy hospital office line within a short time, which may have limited the quality of the interview.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The absence of properly registered protocols, failure of authors to provide protocols, and absence of protocols in some studies clearly affected the quality of the studies.

    “We also found that there was no instance of a properly registered protocol, no author provided their protocol to us, and some studies had no protocol at all, all of which clearly affected the quality of the studies.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Premisecitation

    Professor Hu considered these trends to be due to a lack of education in clinical study methodology and professional ethics in China.

    “Professor Hu, an expert in biomedical statistics at the Academy of Military Medical Science and the vice-president of the Chinese Society of Biomedical Engineering, also referred to these trends and considered them to be due to a lack of education in clinical study methodology and professional ethics in China [22].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] Liao J, Bi S: Clinical studies should be prospective: an interview to biomedical statistical expert professor Hu Lianping (廖杰, 碧石. 临床研究要有前瞻性——访生物统计学专家胡良平教授. 中国医学论坛报). China Medical Tribune 2005.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Scientific misconduct occurs in many disciplines and many countries in the world, including Japan, the UK and the USA, as well as many other countries.

    “Finally, and most importantly, it cannot be emphasized enough that scientific misconduct occurs in many disciplines and many countries in the world, including Japan, the UK and the USA, as well as many other countries [23–28].”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [23] DuBois JM, Anderson EE, Chibnall J, Carroll K, Gibb T, Ogbuka C, Rubbelke T. Understanding research misconduct: a comparative analysis of 120 cases of professional wrongdoing. Account Res. 2013;20(5–6):320–38. source

      Not checked in the free run.

    • cites [24] Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One. 2009;4(5):e5738. source

      Not checked in the free run.

    • cites [25] A consensus statement on research misconduct in the UK. BMJ 2012, 344:e1111. source

      Not checked in the free run.

    • cites [26] McCarthy M. Independent board needed to promote research integrity, US panel says. BMJ. 2017;357:j1879. source

      Not checked in the free run.

    • cites [27] Bolland MJ, Avenell A, Gamble GD, Grey A. Systematic review and statistical analysis of the integrity of 33 randomized controlled trials. Neurology. 2016;87(23):2391–402. source

      Not checked in the free run.

    • cites [28] Carlisle JB. The analysis of 168 randomised controlled trials to test data integrity. Anaesthesia. 2012;67(5):521–37. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Premisecitation

    Some individuals suggest clinical trial fraud is "an open secret" in China.

    “Some individuals suggest clinical trial fraud is “an open secret” in China [13].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] Woodhead M. 80% of China's clinical trial data are fraudulent, investigation finds. BMJ. 2016;355:i5396. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The academic competition of "publish or perish" may also be at play because the number of publications carries substantial weight in obtaining an advanced degree, job application, promotion and grant application.

    “The academic competition of “publish or perish” may also be at play here as the number of publications carries substantial weight in obtaining an advanced degree, job application, promotion and grant application [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] Qiu J. Publish or perish in China. Nature. 2010;463(7278):142–3. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The results may not be generalizable to RCTs from China in other specialties or RCTs identified from China found in other databases.

    “First, we identified only RCTs that examined new generation antidepressants and antipsychotics in the CNKI, and the results may not be generalizable to RCTs from China in other specialties or RCTs identified from China found in other databases.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The results may be different if the search for relevant Chinese trials is limited to selected journals such as the Core Journals from the Beijing University Library or China Scientific and Technical Papers and Citations Database.

    “The CNKI lacks an index of recommended journals, such as the Core Journals from the Beijing University Library or China Scientific and Technical Papers and Citations Database, and the results may be different if we limit the search for the relevant Chinese trials to these selected journals.”
    Quote in the paperNot a substantial claim
53 of 53

  • The authors state this as an assumption.

    The studies for which the authors declined interviews had an unclear risk of bias.

    “Assuming that the studies for which the authors declined interviews had an unclear risk, the proportion of RCTs at low risk of bias in both sequence generation and allocation concealment was 9.2% (10/109, 95% confidence interval [CI]: 5.0 to 16.2).”

  • The paper relies on this without support.

    Telephone interviews with study authors yield accurate information about the methodological conduct of their trials.

    “It is therefore imperative to investigate the quality of RCTs recently published in China by conducting direct interviews with the study authors.”

  • The authors state this as an assumption.

    10–20% of the trials identified in the CNKI are at low risk for both sequence generation and allocation concealment.

    “Assuming 10–20% of the trials identified in the CNKI to be at low rsik for both sequence generation and allocation concealment, 100–300 trials can be identified in the CNKI in recent years.”

  • The authors state this as an assumption.

    Authors who explicitly refused the interview are at unclear risk of bias.

    “Excluding the uncontactable authors but including the authors who explicitly refused the interview in the denominators and assuming them to be at unclear risk of bias, the proportion of RCTs at low risk of bias in random allocation was 9.2% (10/109, 95% CI: 5.0% to 16.2%).”

  • The paper relies on this without support.

    Telephone interview responses accurately reflect the actual conduct of the trials.

    “The risk of bias assessment was initially based on the published reports and then on telephone interviews with the authors.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Conflicting counts for low-risk intention-to-treat analysis

    Minor

    The table lists 52 successfully interviewed reports at low risk of bias for intention-to-treat analysis, while the results text says 51. The text's counts of 51 low, three unclear, and four high also sum to the 58 interviewed reports.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Higgins JP, Green S (eds.): Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011] Available from http://handbook-5-1.cochrane.org/ ; 2011.

Cited by

Sorted by citations

Related

Authors
Tong, Z., Li, F., Ogawa, Y., Watanabe, N., & Furukawa, T. A.
Affiliations
Kyoto University (JP)
Corresponding author
Furukawa, Toshi A.
ORCID
Venue
BMC Medical Research Methodology
Publisher
Springer Science and Business Media LLC
Published
2018-09-24
Volume
18
Issue
1
Pages
96-96
ISSN
1471-2288 (ISSN-L)
DOI
10.1186/s12874-018-0554-2
PMC
PMC6154421
OpenAlex
W2892022362
Semantic Scholar
8c11600c47a80004134d5cd3753461bf322168ff
Cited by
61
References
32
Citations by year
2026: 8 · 2025: 2 · 2024: 8 · 2023: 17 · 2022: 7 · 2021: 11 · 2020: 2 · 2019: 6
FWCI
3.62
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Japan Society for the Promotion of Science (17K19808)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Pharmacology › Pharmaceutical industry and healthcare
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords
randomized controlled trials, trial quality, risk of bias, antipsychotic drugs, CNKI, sequence generation, allocation concealment, methodological quality
MeSH terms
Data Accuracy, Antidepressive Agents, China, Humans, Interviews as Topic, Publications, Research Design, Antipsychotic Agents, Randomized Controlled Trials as Topic, Databases, Factual, Total Quality Management

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review