Limitations are not properly acknowledged in the scientific literature

Abstract

Limitations are important to understand for placing research findings in context, interpreting the validity of the scientific work, and ascribing a credibility level to the conclusions of published research. This goes beyond listing the magnitude and direction of random and systematic errors and validity problems. Acknowledgment of limitations requires an interpretation of the meaning and influence of errors and validity problems on the published findings. An examination of the full-text files of the first 50 articles published in 2005 in the six most-cited research journals and in two recently launched leading open-access journals showed that only 67 articles (17%) used at least one word denoting limitations in the context of the presented scientific work. Only four articles (1%) used the word limitation in their abstract; none referred to limitations of the present work that materially affected conclusions. Only five articles had a separate section on limitations. Conversely, 243 articles (61%) used words detected by the roots error, valid, bias, reproducib, or false and 289 articles (72%) used words with the root importan. Among the 25 top-cited journals' instructions to the authors and editorial policies, only one encourages discussion of limitations; importance, novelty, and lack of error are typically encouraged. Limitations should be better covered and discussed in research articles. To facilitate this, journals should give better guidance and promote the discussion of…