Critical appraisal of quantitative PCR results in colorectal cancer research: Can we rely on published qPCR results?
Open accessAbstract
The use of real‐time quantitative polymerase chain reaction (qPCR) in cancer research has become ubiquitous. The relative simplicity of qPCR experiments, which deliver fast and cost‐effective results, means that each year an increasing number of papers utilizing this technique are being published. But how reliable are the published results? Since the validity of gene expression data is greatly dependent on appropriate normalisation to compensate for sample‐to‐sample and run‐to‐run variation, we have evaluated the adequacy of normalisation procedures in qPCR‐based experiments. Consequently, we assessed all colorectal cancer publications that made use of qPCR from 2006 until August 2013 for the number of reference genes used and whether they had been validated. Using even these minimal evaluation criteria, the validity of only three percent (6/179) of the publications can be adequately assessed. We describe common errors, and conclude that the current state of reporting on qPCR in colorectal cancer research is disquieting. Extrapolated to the study of cancer in general, it is clear that the majority of studies using qPCR cannot be reliably assessed and that at best, the results of these studies may or may not be valid and at worst, pervasive incorrect normalisation is resulting in the wholesale publication of incorrect conclusions. This survey demonstrates that the existence of guidelines, such as MIQE, is necessary but not sufficient to address this problem and suggests that the scientific community should examine its responsibility and be aware of the implications of these findings for current and future research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Poorly optimized qPCR reactions can result in artifactual Cq values and misinterpreted data that are difficult or impossible to reproduce.
“Poorly optimized reactions can result in artifactual Cq values and misinterpreted data that are difficult or even impossible to reproduce2, 3.”
From Droplet Digital PCR versus qPCR for gene expression analysis with low abundant targets: from variable nonsense to publication quality data (2017) · cites this paper as [2]
The Introduction says qPCR parameters must be optimized for meaningful, reproducible results and that inefficient assays increase variability. It also discusses variations in Cq values, while the Discussion warns that inadequate methods can lead to incorrect conclusions. Our copy does not specifically say that poor optimization produces artifactual Cq values or makes the resulting data impossible to reproduce; it says many published results cannot be reliably assessed. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Analyzing real-time PCR data by the comparative CT method
Schmittgen, T. D., & Livak, K. J. (2008) · Nature Protocols · doi:10.1038/nprot.2008.73 · Cited by 26,190 · see citing works
Vandesompele, J., De Preter, K., Pattyn, F., Poppe, B., Van Roy, N., De Paepe, A., & Speleman, F. (2002) · Genome Biology · doi:10.1186/gb-2002-3-7-research0034 · Cited by 20,038 · see citing works
The MIQE guidelines: minimum information for publication of quantitative real-time PCR experiments
Bustin, S. A., Benes, V., Garson, J. A., Hellemans, J., Huggett, J., Kubista, M., Mueller, R., Nolan, T., Pfaffl, M. W., Shipley, G. L., et al. (2009) · Clinical Chemistry · doi:10.1373/clinchem.2008.112797 · Cited by 16,303 · see citing works
Self-citationBustin, S. A. (2000) · Journal of Molecular Endocrinology · doi:10.1677/jme.0.0250169 · Cited by 3,844 · see citing works
Self-citationBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · see citing works
Guideline to reference gene selection for quantitative real-time PCR
Radonić, A., Thulke, S., Mackay, I. M., Landt, O., Siegert, W., & Nitsche, A. (2003) · Biochemical and Biophysical Research Communications · doi:10.1016/j.bbrc.2003.11.177 · Cited by 1,645 · see citing works
Housekeeping genes as internal standards: use and limits
Thellin, O., Zorzi, W., Lakaye, B., Borman, B. D., Coumans, B., Hennen, G. P., Grisar, T. M., Igout, A., & Heinen, E. (1999) · Journal of Biotechnology · doi:10.1016/s0168-1656(99)00163-7 · Cited by 1,534 · see citing works
Simultaneous Amplification and Detection of Specific DNA Sequences
Higuchi, R., Dollinger, G. D., Walsh, P. S., & Griffith, R. L. (1992) · Nature Biotechnology · doi:10.1038/nbt0492-413 · Cited by 1,130 · see citing works
A novel and universal method for microRNA RT-qPCR data normalization
Mestdagh, P., Van Vlierberghe, P., De Weer, A., Muth, D., Westermann, F., Speleman, F., & Vandesompele, J. (2009) · Genome Biology · doi:10.1186/gb-2009-10-6-r64 · Cited by 924 · see citing works
Control Selection for RNA Quantitation
Suzuki, T., Higgins, P. J., & Crawford, D. R. (2000) · BioTechniques · doi:10.2144/00292rv02 · Cited by 878 · see citing works
How to do successful gene expression analysis using real-time PCR
Derveaux, S., Vandesompele, J., & Hellemans, J. (2009) · Methods · doi:10.1016/j.ymeth.2009.11.001 · Cited by 791 · see citing works
de Kok, J. B., Roelofs, R., Giesendorf, B. A. J., Pennings, J. L. A., Waas, E. T., Feuth, T. B., Swinkels, D. W., & Span, P. N. (2004) · Laboratory Investigation · doi:10.1038/labinvest.3700208 · Cited by 580 · see citing works
Warrington, J. A., NAIR, A., Mahadevappa, M., et al. (2000) · Physiological Genomics · doi:10.1152/physiolgenomics.2000.2.3.143 · Cited by 541 · see citing works
Standardized determination of real-time PCR efficiency from a single reaction set-up
Tichopád, A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gng122 · Cited by 456 · see citing works
Why the need for qPCR publication guidelines?—The case for MIQE
Bustin, S. A. (2009) · Methods · doi:10.1016/j.ymeth.2009.12.006 · Cited by 374 · see citing works
Self-citationThe need for transparency and good practices in the qPCR literature
Bustin, S. A., Benes, V., Garson, J., Hellemans, J., Huggett, J., Kubista, M., Mueller, R., Nolan, T., Pfaffl, M. W., Shipley, G., et al. (2013) · Nature Methods · doi:10.1038/nmeth.2697 · Cited by 281 · see citing works
Self-citationMeyer, S. U., Pfaffl, M. W., & Ulbrich, S. E. (2010) · Biotechnology Letters · doi:10.1007/s10529-010-0380-z · Cited by 212 · see citing works
Mazzoccoli, G., Panza, A., Valvano, M. R., Palumbo, O., Carella, M., Pazienza, V., Biscaglia, G., Tavano, F., Sebastiano, P. D., Andriulli, A., et al. (2011) · Chronobiology International · doi:10.3109/07420528.2011.615182 · Cited by 132 · see citing works
Housekeeping genes; expression levels may change with density of cultured cells
Greer, S. N., Honeywell, R., Geletu, M., Arulanandam, R., & Raptis, L. H. (2010) · Journal of Immunological Methods · doi:10.1016/j.jim.2010.02.006 · Cited by 128 · see citing works
Inhibition of real-time RT–PCR quantification due to tissue-specific contaminants
Tichopád, A., Didier, A., & Pfaffl, M. W. (2003) · Molecular and Cellular Probes · doi:10.1016/j.mcp.2003.09.001 · Cited by 126 · see citing works
Design and Optimization of Reverse-Transcription Quantitative PCR Experiments
Tichopád, A., Kitchen, R., Riedmaier, I., Becker, C., Ståhlberg, A., & Kubista, M. (2009) · Clinical Chemistry · doi:10.1373/clinchem.2009.126201 · Cited by 104 · see citing works
‘Desperate house genes’: the dramatic example of hypoxia
Caradec, J., Sirab, N., Keumeugni, C., et al. (2010) · British Journal of Cancer · doi:10.1038/sj.bjc.6605573 · Cited by 91 · see citing works
Epigenetic silencing of EphA1 expression in colorectal cancer is correlated with poor survival
Herath, N. I., Doecke, J. D., Spanevello, M., et al. (2009) · British Journal of Cancer · doi:10.1038/sj.bjc.6604970 · Cited by 74 · see citing works
Tumour CD133 mRNA expression and clinical outcome in surgically resected colorectal cancer patients
Artells, R., Moreno, I., Díaz, T., et al. (2009) · European Journal of Cancer · doi:10.1016/j.ejca.2009.11.003 · Cited by 71 · see citing works
Sørby, L. A., Andersen, S. N., Bukholm, I., & Jacobsen, M. (2010) · Journal of Experimental & Clinical Cancer Research · doi:10.1186/1756-9966-29-144 · Cited by 56 · see citing works
Dijkstra, J. R., Mekenkamp, L. J. M., Teerenstra, S., De Krijger, I., & Nagtegaal, I. D. (2012) · Journal of Cellular and Molecular Medicine · doi:10.1111/j.1582-4934.2011.01467.x · Cited by 53 · see citing works
Self-citationSelection of an internal control gene for quantitation of mRNA in colonic tissues.
Tsuji, N., Kamagata, C., Furuya, M. H., et al. (2003) · PubMed · Cited by 45 · see citing works
Koga, Y., Yasunaga, M., Moriya, Y., Akasu, T., Fujita, S., Yamamoto, S., Kozu, T., Baba, H., & Matsumura, Y. (2008) · Cancer Science · doi:10.1111/j.1349-7006.2008.00954.x · Cited by 44 · see citing works
Differential CXC Receptor Expression in Colorectal Carcinomas
Rubie, C., Kollmar, O., Frick, V. O., et al. (2008) · Scandinavian Journal of Immunology · doi:10.1111/j.1365-3083.2008.02163.x · Cited by 40 · see citing works
Frequent genomic loss at chr16p13.2 is associated with poor prognosis in colorectal cancer
Andersen, C. L., Lamy, P., Thorsen, K., Kjeldsen, E., Wikman, F. P., Villesen, P., Øster, B., Laurberg, S., & Orntoft, T. F. (2010) · International Journal of Cancer · doi:10.1002/ijc.25841 · Cited by 39 · see citing works
CXC receptor-4 mRNA silencing abrogates CXCL12-induced migration of colorectal cancer cells
Rubie, C., Frick, V. O., Ghadjar, P., et al. (2011) · Journal of Translational Medicine · doi:10.1186/1479-5876-9-22 · Cited by 33 · see citing works
Ceelen, L. M., Craene, J. D., & Spiegelaere, W. D. (2013) · Clinical Chemistry · doi:10.1373/clinchem.2013.209478 · Cited by 25 · see citing works
What's Different about Women's Health?
Gronowski, A. M., Manson, J. E., Mardis, E. R., Mora, S., & Spong, C. Y. (2013) · Clinical Chemistry · doi:10.1373/clinchem.2013.216598 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Taylor, S. C., Laperriere, G., & Germain, H. (2017) · Scientific Reports · doi:10.1038/s41598-017-02217-x · Cited by 626 · see citing works
“(2) Presuming good, sigmoidal amplification curves are obtained with single melt curve peaks for each primer pair, run the duplicate samples from the qPCR reactions on a gel to assess the molecular weight and identity of the amplicon and sequence if necessary.”
BackgroundTaylor, S. C., Carbonneau, J., Shelton, D. N., & Boivin, G. (2015) · Journal of Virological Methods · doi:10.1016/j.jviromet.2015.08.014 · Cited by 142 · see citing works
Bustin, S., & Nolan, T. (2017) · European Journal of Clinical Investigation · doi:10.1111/eci.12801 · Cited by 113 · see citing works
Self-citationImproving the reliability of peer-reviewed publications: We are all in it together
Bustin, S. A., & Nolan, T. (2016) · Biomolecular Detection and Quantification · doi:10.1016/j.bdq.2015.11.002 · Cited by 22 · see citing works
Self-citation
- Authors
- Dijkstra, J. R., van Kempen, L. C., Nagtegaal, I. D., & Bustin, S. A.
- Affiliations
- Radboud University Medical Center (NL)
- Anglia Ruskin University (GB)
- Corresponding author
- Dijkstra, J.R.
- Venue
- Molecular Oncology
- Publisher
- Wiley
- Published
- 2014-01-02
- Volume
- 8
- Issue
- 4
- Pages
- 813-818
- ISSN
- 1574-7891 (ISSN-L)
- 1878-0261
- PMC
- PMC5528534
- OpenAlex
- W2105131625
- Cited by
- 59
- References
- 33
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 3 · 2023: 3 · 2022: 3 · 2021: 4 · 2020: 2 · 2019: 6 · 2018: 4 · 2017: 8 · 2016: 3 · 2015: 8 · 2014: 6
- FWCI
- 3.42
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Molecular Oncology (publishedVersion, other-oa)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Radboud Repository (Radboud University) (publishedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Anglia Ruskin Research Online (Anglia Ruskin University) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Molecular Biology Techniques and Applications
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- real-time quantitative PCR, gene expression normalization, reference genes, colorectal cancer, research reproducibility, MIQE guidelines
- MeSH terms
- Humans, Prognosis, Publications, Biomarkers, Tumor, Colorectal Neoplasms, Reproducibility of Results, Biomedical Research, Real-Time Polymerase Chain Reaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
