CoChat
Sign Up
Cited by 7Academic integrity and plagiarismMeta-analysis and systematic reviewsBenford’s Law and Fraud Detection

Fraudulent studies are undermining the reliability of systematic reviews: on the prevalence of problematic images in preclinical depression studies

Open access
Berrío, Jenny P., Kalliokoski, Otto·2025·FEBS Letters
DOI 10.1002/1873-3468.70077

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Systematic reviews are considered by many to constitute the highest level of scientific evidence. However, the methods used in a systematic review for combining information from multiple studies are predicated on all of the reports being truthful. For a systematic review of preclinical studies of depression, we found that potentially fraudulent studies-studies featuring problematic images suggestive of gross error or manipulation-were both common and capable of biasing our findings. The prevalence of problematic studies (we had concerns with 19% of all studies with images) and our inability to find a simple pattern for identifying them undermine systematic reviews within our research field. We suspect that this is symptomatic of a broader problem that needs immediate addressing.

Integrity signals

Registry

Crossref · 10.1002/1873-3468.70077

Cites

97 of 99 references verified0 citation errors · 2 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

16 of 37 cited sources checked · 6 supported · 1 flagged · accuracy not yet measured

  • Findingcitation

    Checklists and protocols for assessing evidence quality do not single out problematic reports.

    “Checklists and protocols for gauging the quality of evidence in a systematic fashion 8,9 also do not single out these problematic reports.”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [8] 8. Hooijmans, C. R. et al. SYRCLE's risk of bias tool for animal studies. BMC Med. Res. Methodol. 14, 43 (2014). source

      The Results section (‘Use of SYRCLE’s RoB tool’; page unavailable) says the tool assigns each study item a low, high, or unclear risk-of-bias judgment. It also describes presenting results for individual studies and excluding high-risk studies in a sensitivity analysis. That is contrary to the broad claim that systematic quality-assessment tools do not single out problematic reports, though the citing sentence does not specify what ‘these’ reports are. (LLM, conf=0.79)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [9] 9. Higgins, J. P. et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343, d5928 (2011). source

      The risk of bias tool and Presentation of assessments sections establish that this paper describes a systematic method for assessing trials. They do not establish that such methods fail to single out the citing sentence’s unspecified “problematic reports”; the tool instead assigns high, low, or unclear risk-of-bias judgments to individual trials. No page number is available in our copy. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
  • Interpretationreasoning

    A small number of people with flexible morals can disproportionately pollute the scientific literature.

    “This is how a small number of people 40 with flexible morals can pollute the scientific literature disproportionately.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [40] 40. Fanelli, D. How many scientists fabricate and falsify research? A systematic review and meta- analysis of survey data. PloS one 4, e5738 (2009). source
  • Findingcitation

    Their estimate of potentially fraudulent papers aligns with recent discoveries.

    “Secondly, our estimate for potentially fraudulent papers aligns well with recent discoveries.”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [15] 15. Bik, E. M., Casadevall, A. & Fang, F. C. The prevalence of inappropriate image duplication in biomedical research publications. MBio 7, 10.1128/mbio. 00809-00816 (2016). source
    • cites [45] 45. Cho, D.-Y., Bishop, J., Grayson, J. & Woodworth, B. A. Inappropriate image duplications in rhinology research publications. International Forum of Allergy & Rhinology 14, 119-122 (2023). source

      The Discussion (page unavailable) reports that approximately 12% of the sampled rhinology articles may contain inappropriate image duplications. The Methods (page unavailable) say the more serious types are more likely to result from misconduct, supporting the reference to potentially fraudulent papers. But the cited paper cannot establish whether the citing authors’ unspecified estimate aligns with its finding. (LLM, conf=0.79)

      Experimental — accuracy not yet measured
    • cites [46] 46. Bucci, E. M. Automatic detection of image manipulations in the biomedical literature. Cell Death Dis. 9, 400 (2018). source

      In “Investigation of a random set of Open Access papers” (page unavailable), Bucci reports suspected image manipulation in 78 of 1,364 papers (5.7%), providing a finding with which the citing authors could compare their estimate. The same section cautions that identified manipulations are not necessarily connected to misconduct or fraud, so the source does not establish a rate of potentially fraudulent papers. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Findingown data

    Reports with problematic images were not cited less often than reports without such problems.

    “Counter to commonly held beliefs, reports with problematic images were not cited less or published in lower-impact journals, nor were their authors isolated to any specific geographic area.”
    • Also stated in: § Bibliometrics · plain
      “On average, the studies featuring problematic images were cited as often as were the studies without issues.”
    Quote in the paperSubstantial claim
  • Findingown data

    Among 1,035 screened peer-reviewed reports, 19% displayed research data as problematic images.

    “In a sample of 1,035 studies, we found that 19 % of peerreviewed reports displayed data in the form of problematic images.”
    • Also stated in: § Overview · hedged
      “We flagged 112 papers with problematic images, suggesting that one in five papers within the research field (19 %) suffer from these issues (Figure 2).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The screened publications are representative of the preclinical depression research field.
  • Classified as probably not a substantial claim. 35
  • May combine two claims. 23
  • The quote only partly states this claim. 2

  • Classified as probably not a substantial claim.

    Premisecitation

    A rat exposed to constantly shifting but invariably stressful conditions daily will develop something akin to major depression.

    “The theory is that a rat exposed to constantly-shifting-but-invariably-stressful conditions on a daily basis will develop something akin to a major depression 11 .”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [11] 11. Katz, R. J., Roth, K. A. & Carroll, B. J. Acute and chronic stress effects on open field activity in the rat: implications for a model of depression. Neurosci. Biobehav. Rev. 5, 247-251 (1981). source

      The Description of Second Factor (page not available) describes varied, unpredictable stressors, but says they were separated by one to two days rather than administered daily. The Abstract and Discussion (pages not available) frame the resulting changes in open-field behavior as a possible model with parallels to clinical depression, not as a demonstration that rats develop major depression. (LLM, conf=0.87)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation

    The unpredictable chronic stress model in laboratory rats is used to model depression.

    “Our systematic review concerned unpredictable chronic stress, and its use in modelling depression in laboratory rats 10 .”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [10] 10. Berrio, J. P., Kalliokoski, O. & Hestehave, S. Reliability of sucrose preference testing following short or no food and water deprivation. A Systematic Review and Meta-analysis of rat models of chronic unpredictable stress. bioRxiv, 2023.2002. 2022.529490 (2023). source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    China has struggled with poorly aligned researcher incentives that foster a problematic publication culture in certain research environments.

    “China has admittedly struggled with poorly aligned researcher incentives 33 fostering a problematic publication culture in certain research environments 34 .”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [33] 33. Quan, W., Chen, B. & Shu, F. Publish or impoverish: An investigation of the monetary reward system of science in China (1999-2016). Aslib Journal of Information Management 69, 486-502 (2017). source
    • cites [34] 34. Hvistendahl, M. China's publication bazaar. Science 342, 1035-1039 (2013). source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The authors suspect that most researchers who agree research fraud is rare have not encountered it personally.

    “We would suspect that the reason why most researchers would agree with the statement that research fraud is rare, all the while data seems to point to the contrary 35,36 , is that they have not encountered it personally 37 .”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites [35] 35. Fang, F. C., Steen, R. G. & Casadevall, A. Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci 109, 17028-17033 (2012). source

      The Discussion reports that fraud accounts for nearly half of retractions, but it does not discuss whether researchers consider fraud rare or whether personal experience explains that view. Searches for personal encounters, perceptions of rarity, and survey evidence found no passage addressing the explanation attached to citation 37. (LLM, conf=0.75)

      Experimental — accuracy not yet measured
    • cites [36] 36. Xie, Y., Wang, K. & Kong, Y. Prevalence of research misconduct and questionable research practices: A systematic review and meta-analysis. Sci. Eng. Ethics 27, 41 (2021). source
    • cites [37] 37. Wells, J. Observing and reporting suspected misconduct in biomedical research. The Gallup Organization and The Office of Research Integrity (2008).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    The few similar investigations have produced similar prevalence estimates for problematic images, though this study's estimate is among the highest.

    “There are few investigations that have employed similar approaches; however, the ones we know of 15,[45][46][47][48] have produced similar estimates for the prevalence of reports featuring problematic images (although we appear to have the dubious honor of having produced one of the highest estimate so far).”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [15] 15. Bik, E. M., Casadevall, A. & Fang, F. C. The prevalence of inappropriate image duplication in biomedical research publications. MBio 7, 10.1128/mbio. 00809-00816 (2016). source
    • cites [45] 45. Cho, D.-Y., Bishop, J., Grayson, J. & Woodworth, B. A. Inappropriate image duplications in rhinology research publications. International Forum of Allergy & Rhinology 14, 119-122 (2023). source
    • cites [46] 46. Bucci, E. M. Automatic detection of image manipulations in the biomedical literature. Cell Death Dis. 9, 400 (2018). source

  • May combine two claims.

    Premisecitation

    Journal retractions happen slowly, and author protests can sometimes prevent them entirely.

    “Journal retractions happen at a glacial speed, and author protests can sometimes be enough to prevent them wholesale 59,60 .”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [59] 59. Oransky, I. Retractions are increasing, but not enough. Nature 608, 9-9 (2022). source

      In “Retractions are increasing, but not enough” (p. 1), Oransky says retracting a paper is slow and often takes years. The paper also says lawyers’ threats can prevent corrections to the literature, but it does not say that author protests can prevent journal retractions; it describes those legal threats as rarely succeeding. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [60] 60. Besançon, L., Bik, E., Heathers, J. & Meyerowitz-Katz, G. Correction of scientific literature: too little, too late! PLoS Biol. 20, e3001572 (2022). source

      The Abstract describes a “glacial pace” of scientific correction and says retraction may occur months after publication, supporting the claim that retractions are slow. Neither the Abstract nor the Box 1 passages say that author protests can prevent a retraction entirely; threats from critiqued authors are not evidence of that outcome. (LLM, conf=0.85)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S2 (checkable). The quote only partly states this claim.

    Premisereasoning

    The self-correcting nature of science is at best torturously slow.

    “It can be shown that the”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites [56] 56. Grimes, D. R. Is biomedical research self-correcting? Modelling insights on the persistence of spurious science. bioRxiv, 2023.2007. 2017.549436 (2023). source
    • cites [57] 57. Nissen, S. B., Magidson, T., Gross, K. & Bergstrom, C. T. Publication bias and the canonization of false facts. Elife 5, e21451 (2016). 58 Cochrane Database of Systematic Reviews: editorial policies. Cochrane Library (2023). source

      The Discussion section, “What do we do about the problem of publication bias?” (page unavailable), says science can self-correct after false claims become accepted as facts. It does not establish that correction is “torturously slow”; it says inconsistencies soon begin to accumulate, without specifying how long correction takes. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The researchers were cleared on all five counts of suspected misconduct.

    “The researchers were cleared on all five counts of suspected misconduct 44 .”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [44] 44. J. Affect. Disord. 311, 641 (2022). source

      The ‘Investigating suspicious citations’ section (pp. 6, 9) describes five authors with suspicious citation patterns, not five counts of misconduct. Searches for clearance, allegations, and misconduct investigations found no passage reporting that researchers were cleared. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    In a systematic review of preclinical depression studies, potentially fraudulent studies were common and biased the review's findings.

    “For a systematic review of preclinical studies of depression, we found that potentially fraudulent studies were not only common but also that they biased the findings of the review.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Systematic review methods that combine multiple studies require all included reports to be truthful.
    • Rests on an assumption: The screened publications are representative of the preclinical depression research field.

  • May combine two claims.

    Findingown data

    Journal renown and citation by other researchers do not reduce the chance that a paper has problematic images.

    “Focusing on the renown of the journal publishing the paper, or seeing whether the findings were cited by other researchers, does not seem to reduce the chances that a paper presents with problematic images.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Of 1,035 screened papers, 132 met all requirements for inclusion in the systematic review.

    “Of the 1,035 papers we screened, 132 met all of our requirements for being included in our systematic review.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The authors scanned 1,035 peer-reviewed publications, of which 588 presented research data in image form.

    “In our investigation, we scanned 1,035 peer-reviewed publications, but only 588 papers presented research data in image form (excluding line drawings like plots, graphs, infographics, and schematics).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Micrograph issues were frequent, including histology (28), immunohistochemistry (18), and immunofluorescence (13) images.

    “Issues with micrographs were also frequent, with histology images (28 studies), immunohistochemistry images (18 studies), and immunofluorescence images (13 studies) being particularly numerous.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The flagged papers had higher average effect sizes than non-flagged papers, with an average difference of 0.81 standard deviations (Q1=4.57, p=0.03).

    “Overall, the flagged papers presented with effect sizes that were, on average, higher than that of papers that we had not flagged (Average difference: 0.81 standard deviations; Cochran's Q test used for sub-group analysis: Q1 = 4.57, p = 0.03), but not so much so that they would look out of place (Figure 3).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The authors' investigation found image issues in one in five peer-reviewed papers, and they believe the majority were in parts or fully doctored.

    “Our investigation found image issues in onein-five peer-reviewed papers, the majority of which we believe to be, in parts or fully, doctored.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The screened publications are representative of the preclinical depression research field.

  • May combine two claims.

    Findingown data

    Papers with problematic images cannot be distinguished from papers using simple metrics; neither journal impact factor (median 3.3) nor yearly citations (median 3.7) can identify them.

    “A simple bibliometric analysis suggests that papers with problematic images cannot be distinguished from papers using simple metrics.Neither the impact factor of the journals that publish these papers (median: 3.3), nor the yearly number of citations they accrue (median: 3.7), can be used to identify problematic and potentially fraudulent studies.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Ten included papers were flagged with image issues; six had Type II (3) or Type III (3) issues.

    “Ten of these were flagged with image issues; six of them with Type II (3) or Type III (3) issues.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Of 24 studies with analyzable images and one or more Danish authors, seven had problematic images (risk ratio 1.98).

    “Of 24 studies containing analyzable images and with one or more Danish authors, seven were found to have problematic images (risk ratio: 1.98).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The retraction was requested by the journal's Editor-in-Chief after they unsuccessfully sought an explanation from the purported authors.

    “The retraction was requested by the journal's Editor-in-Chief, after they had unsuccessfully sought the (purported) authors for an explanation 42 .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [42] 42. Pei, G., Xu, L., Huang, W. & Yin, J. Retraction notice to" The protective role of microRNA-133b in restricting hippocampal neurons apoptosis and inflammatory injury in rats with depression by suppressing CTGF"[Int. Immunopharmacol. 78 (2020) 106076]. Int. Immunopharmacol. 117, 109872-109872 (2023). source

  • May combine two claims.

    Findingreasoning

    Impartial experts were not consulted, the papers remain uncorrected, and the ruling cannot be appealed.

    “Impartial experts were not consulted, the papers remain uncorrected, and the ruling cannot be appealed.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Reports with problematic images had higher-than-average effect sizes, but not the highest.

    “Sure, we found that reports with problematic images present higherthan-average effect sizes, but they were not the highest.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Findingreasoning

    No simple shortcuts exist for determining a report's reliability.

    “Simple shortcuts to determining the reliability of a report do not seem to exist.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    An anonymous user provided animations demonstrating some overlapping images listed on YouTube.

    “In a number of cases, anonymous user Illex illecebrosus provided animations demonstrating some of the overlapping images we had listed on the YouTube channel ZeebaTV.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In preclinical studies, most problematic studies (and among them, presumably fraudulent) are mundane.

    “Yet, what we see in our investigation is that -at least within preclinical studies -most problematic studies (and among them, we speculate, the fraudulent) are mundane.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Higher-impact journals appear less likely to publish problematic images, but publication in a higher-impact journal does not guarantee integrity.

    “Whereas journals with a higher impact factor appear to be less likely to publish a study with problematic images 15 , our investigation suggests that being published in a higher-impact journal is no guarantee of a paper's integrity.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [15] 15. Bik, E. M., Casadevall, A. & Fang, F. C. The prevalence of inappropriate image duplication in biomedical research publications. MBio 7, 10.1128/mbio. 00809-00816 (2016). source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    A typical example is two sets of western blots allegedly imaged from the same membrane, which was clearly not possible.

    “A typical example is a situation where two sets of western blots were allegedly imaged from the same membrane, but where this was clearly not possible.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The studies from a Danish research group were reported to the Danish Board on Research Misconduct.

    “The studies originating from a Danish research group were reported to the Danish Board on Research Misconduct.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Only one publication in the analyses had never been cited.

    “Only one publication in our analyses had never been cited.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    One study reported two separate experiments, both contributing evidence to the primary research question.

    “One study reported on two separate experiments, both contributing evidence to our primary research question.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Anonymous users made additional discoveries in five cases where the authors reported issues.

    “Where we reported on issues with papers, anonymous users (users: Hoya camphorifolia, Mycosphaerella arachidis, Schinia honesta, and Uromycladium fusisporum) made additional discoveries in five cases.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    A small number of images had issues that did not fit the three levels of the Bik scale, so an 'other' category was added.

    “We found a small number of images with issues that did not fit well in the three levels of the”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. The quote only partly states this claim.

    Findingown data

    A list of 447 'clean' publications with no image remarks was constructed.

    “A list of 447”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The misconduct investigation consisted of contacting the researchers and asking about the images, which they insisted were not duplicated.

    “The investigation consisted of contacting the researchers and asking about the images, which they insisted were not duplicated.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Two papers already had issues reported on PubPeer by an anonymous user.

    “Two papers were found already to have issues reported on PubPeer by anonymous user Hoya camphorifolia.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The prevalence of problematic studies and the absence of a simple pattern for identifying them undermine the validity of systematic reviews in this research field.

    “The sheer prevalence of problematic studies, and the fact that we could not find a simple pattern for identifying them, undermines the validity of systematic reviews within our research field.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Systematic review methods that combine multiple studies require all included reports to be truthful.
    • Rests on an assumption: The screened publications are representative of the preclinical depression research field.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The divergence from Bik et al. may be because this study focused on publication level while the previous investigation focused on journal level.

    “The diverging results may be an effect of us focusing on a publications level, whereas the previous investigation chose to focus on a journal level.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    In that case, the authors claimed to possess high-resolution scans proving the images were highly similar but not duplicated; the scans were not shared, so the authors consider the issue unresolved.

    “In this case, the authors claimed to possess high-resolution scans of the original western blots, proving that the images were only highly similar but not actually duplicated.These scans were not shared, whereby we consider the issue still to be unresolved.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Authors willing to falsify results are not subject to chance and are guaranteed to obtain desired results, so they can be very productive.

    “Since authors willing to falsify their results are not subject to the cruel whims of chance but always guaranteed to obtain the desired results in their experiments, they can be very productive.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The retracted paper is suspected to be the product of a particularly prolific paper mill.

    “The paper is suspected to be the product of a particularly prolific paper mill 41 .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [41] 41. Smut, C. The scientific sea of miR-and exosome-related knowledge. (2021). <https://forbetterscience.com/2021/09/22/the-scientific-sea-of-mir-and-exosome-related- knowledge/>.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    There is no reason to believe that reports presenting data in image form are more prone to being problematic.

    “There is no reason to believe that reports presenting data in image form are more prone to being problematic.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Although only 29% of studies had clear Type III alterations, many problematic reports in other categories were suspicious.

    “Although only a third (29 %) of the studies featured clear alterations (Type III issues), many of the problematic reports in the other categories were suspicious to us.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Reliability in a research field can be reinstated by identifying a small number of culprits and retracting their work.

    “We can reinstate reliability in a given research field by identifying a small number of culprits and retracting their work.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The sucrose preference test measures the ability to feel joy by studying rats' binge on a sugary solution.

    “We were interested in evaluating the efficacy of studying the effects of this stress protocol using the sucrose preference test -a test that measures the ability to feel joy by studying how rats, when given the chance, will binge on a sugary solution 14 .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] 14. Willner, P., Towell, A., Sampson, D., Sophokleous, S. & Muscat, R. a. Reduction of sucrose preference by chronic unpredictable mild stress, and its restoration by a tricyclic antidepressant. Psychopharmacology 93, 358-364 (1987). source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    China has been implicated as a hotbed for paper mills on multiple occasions.

    “China has been implicated to be such a country on multiple occasions [28][29][30] .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28][29][30]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    The effect sizes in those three studies can be argued to be implausibly large.

    “None of the three was flagged with image issues, although it can be argued that the effect sizes described in the papers are implausibly large.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Manipulating images is more difficult than falsifying tables, fabricating bar charts, or plagiarizing text.

    “If we consider fraudulent studies, for example, it is more difficult to manipulate images than it is to falsify tables, fabricate bar charts, or plagiarize text.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Imagetwin allowed detection of duplications between publications, which is mostly not possible for a human analyst.

    “This allowed us to detect also duplications between publications, something that is mostly not possible for a human analyst.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    In addition to a deluge of potentially fraudulent studies, the authors face institutional unwillingness to deal with potentially fraudulent studies.

    “Outside of facing the problem of a deluge of potentially fraudulent studies, we are also facing a problem with an institutional unwillingness to deal with potentially fraudulent studies.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Cochrane guidelines state that retracted studies should be excluded from systematic reviews.

    “Currently, Cochrane argues in their guidelines that retracted studies should be excluded from systematic reviews 58 .”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [58] 58. <www.cochranelibrary.com/cdsr/editorial-policies#problematic-studies>.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    Image issues were classified using the system developed by Bik et al.

    “The issues we found in published images presenting research data were classified according to the system developed by Bik et al. 15 (refer to Figure 1).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Bik et al. 15

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    A matched-size sample of 112 papers was randomly selected and compared with papers flagged for image issues.

    “A matched-size sample of 112 papers was randomly selected (using a random number generator) and contrasted in a bibliometric analysis with the papers flagged with image issues.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Cross-publication duplication would require much more to go wrong to happen by accident.

    “A lot more would have to go wrong for this to happen by accident.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The classification system was originally developed for images viewed only within their publication context.

    “The classification system was originally developed for images viewed only in the context of the publication they were found in.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bik et al. 15

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

53 of 53

  • The authors state this as an assumption.

    Systematic review methods that combine multiple studies require all included reports to be truthful.

    “A caveat is that the methods used in a systematic review -combining information from multiple studies -are predicated on all of the reports being truthful.”

  • The authors state this as an assumption.

    The frequency of fraudulent studies in systematic reviews and their effect on the evidence base is currently unknown.

    “Currently, we do not know how frequent fraudulent studies are in systematic reviews, or how they affect the resulting evidence base.”

  • The authors state this as an assumption.

    Most guidelines for assessing reliability of peer-reviewed findings assume authors act in good faith.

    “Most guidelines trying to gauge the reliability of findings in a peer-reviewed report assume that the authors are acting in good faith.”

  • The paper relies on this without support.

    The screened publications are representative of the preclinical depression research field.

    “In a sample of 1,035 studies, we found that 19 % of peerreviewed reports displayed data in the form of problematic images.”
    “suggesting that one in five papers within the research field (19 %) suffer from these issues”

  • The paper relies on this without support.

    The chronic unpredictable stress model is widely used across sub-fields.

    “Firstly, the chronic unpredictable stress model is widely used across sub-fields.”

  • The paper relies on this without support.

    Images of experimental setups and stylized images demonstrating a method can be edited without affecting the reliability of the associated report.

    “Images of experimental setups and stylized images demonstrating a method can, obviously, be edited without this affecting the reliability of the associated report.”
6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Cited works that were retracted

Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.

  • Cites a retracted work: 10.1111/jnc.13225

    Major

    The cited work was retracted on 2023-03-21 (notice 10.1111/jnc.15805), before this paper was published (2025-05-21). Its authors could have known.

    Cited work 10.1111/jnc.13225 · retracted 2023-03-21 · notice 10.1111/jnc.15805 · this paper published 2025-05-21

  • Cites a retracted work: 10.1016/j.intimp.2019.106076

    Major

    The cited work was retracted on 2020-01-01 (notice 10.1016/j.intimp.2019.106076), before this paper was published (2025-05-21). Its authors could have known.

    Cited work 10.1016/j.intimp.2019.106076 · retracted 2020-01-01 · notice 10.1016/j.intimp.2019.106076 · this paper published 2025-05-21

  • Cites a retracted work: 10.1136/bmj-2022-071517

    Info

    The cited work was retracted; the date of the notice is not on record.

    Cited work 10.1136/bmj-2022-071517 · retraction date not known · this paper published 2025-05-21

  • Cites a retracted work: 10.1007/s11010-024-05003-2

    Info

    The cited work was retracted; the date of the notice is not on record.

    Cited work 10.1007/s11010-024-05003-2 · retraction date not known · this paper published 2025-05-21

  • Cites a retracted work: 10.1016/j.intimp.2023.109872

    Info

    The cited work was retracted; the date of the notice is not on record.

    Cited work 10.1016/j.intimp.2023.109872 · retraction date not known · this paper published 2025-05-21

  • Cites a retracted work: 10.1016/j.lfs.2023.121418

    Info

    The cited work was retracted; the date of the notice is not on record.

    Cited work 10.1016/j.lfs.2023.121418 · retraction date not known · this paper published 2025-05-21

  • Cites a retracted work: 10.1016/j.jclinepi.2024.111397

    Info

    The cited work was retracted; the date of the notice is not on record.

    Cited work 10.1016/j.jclinepi.2024.111397 · retraction date not known · this paper published 2025-05-21

  • Cites a retracted work: 10.1038/s41467-022-34281-x

    Info

    The cited work was retracted; the date of the notice is not on record.

    Cited work 10.1038/s41467-022-34281-x · retraction date not known · this paper published 2025-05-21

8 of 8

Cites

8 retracted

Flagged first, then by citations

Research works printed but not linked: 2

  1. OranskyI(2014)SAGE Publications busts “peer review and citation ring ” 60 papers retracted.Retraction Watch.
  2. Bik E, 2020, The tadpole paper mill, Sci Integrity Dig

Cited by

Sorted by citations

Related

Authors
Berrío, J. P., & Kalliokoski, O.
Affiliations
University of Copenhagen (DK)
Corresponding author
Kalliokoski, Otto
ORCID
Venue
FEBS Letters
Publisher
Wiley
Published
2025-05-21
Volume
599
Issue
11
Pages
1485-1498
ISSN
  • 0014-5793 (ISSN-L)
  • 1873-3468
DOI
10.1002/1873-3468.70077
OpenAlex
W4410568540
Cited by
7
References
81
Citations by year
2026: 3 · 2025: 1 · 2024: 3
FWCI
0.62
Type
journal article
Language
en
Open access
Licence
Locations
  • FEBS Letters (publishedVersion, other-oa)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research at the University of Copenhagen (University of Copenhagen) (submittedVersion, cc-by)
Funders
National Institute of Food and Agriculture
Topics
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Benford’s Law and Fraud Detection
Keywords
scientific fraud, image editing, questionable research practices, image analysis, research integrity, publication bias
MeSH terms
Systematic Reviews as Topic, Animals, Depression, Humans, Reproducibility of Results, Scientific Misconduct, Prevalence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-01.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-01.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review