Cited by 175Animal testing and alternativesStatistical Methods in Clinical TrialsHealth and Medical Research Impacts

How to design preclinical studies in nanomedicine and cell therapy to maximize the prospects of clinical translation

Open access
Ioannidis, John P. A., Kim, Betty Y. S., Trounson, Alan·2018·Nature Biomedical Engineering

Abstract

The clinical translation of promising products, technologies and interventions from the disciplines of nanomedicine and cell therapy has been slow and inefficient. In part, translation has been hampered by suboptimal research practices that propagate biases and hinder reproducibility. These include the publication of small and underpowered preclinical studies, suboptimal study design (in particular, biased allocation of experimental groups, experimenter bias and lack of necessary controls), the use of uncharacterized or poorly characterized materials, poor understanding of the relevant biology and mechanisms, poor use of statistics, large between-model heterogeneity, absence of replication, lack of interdisciplinarity, poor scientific training in study design and methods, a culture that does not incentivize transparency and sharing, poor or selective reporting, misaligned incentives and rewards, high costs of materials and protocols, and complexity of the developed products, technologies and interventions. In this Perspective, we discuss special manifestations of these problems in nanomedicine and in cell therapy, and describe mitigating strategies. Progress on reducing bias and enhancing reproducibility early on ought to enhance the translational potential of biomedical findings and technologies.

AI snapshot

Written from the paper’s full text.

TL;DR

This Perspective argues that slow clinical translation in nanomedicine and cell therapy is driven largely by bias, suboptimal preclinical design, poor reproducibility, and misaligned incentives, and it proposes design, reporting, standardization, and sharing strategies to improve translatability. It reviews evidence of low replication rates and offers field-specific recommendations for nanomaterials, cell products, models, statistics, and collaboration.

Key findings

  • Reproducibility in preclinical research is low: only 20–25% of 67 preclinical studies considered for translation in oncology, women’s health, and cardiovascular disease could be reproduced; only 11% (6 of 53) of academic oncology drug-target studies could be reproduced; and among the first five highly cited studies in the Reproducibility: Cancer Biology project, only two could be reproduced as originally planned.
  • Suboptimal design is common: in neuroscience, average experimental power is about 20% even with lenient assumptions, and randomization and blinding of investigators who analyse animal experiments are used in less than 20% of studies.
  • Translation remains limited: the paper cites 14 FDA-approved cancer-nanomedicine products and another 35 in clinical trials, and only 8 FDA-approved cell therapies plus 4 EMA-approved cell therapies despite extensive preclinical work.
  • Examples show variable success: CD19 CAR-T therapy for B-cell acute lymphoblastic leukaemia has an ~80% cancer clearance rate but serious side effects and variable patient responses; autologous limbal cell grafting was 80% successful or partially successful at one year and 68% at three years.

Methods

  • Design: a Perspective/commentary, not an original empirical study; it synthesizes prior empirical reproducibility studies, examples, and field-specific literature to propose strategies for preclinical nanomedicine and cell therapy.
  • Data or sample: no new primary dataset; it draws on cited reproducibility checks, meta-research, and selected translational case studies, including trastuzumab emtansine and sipuleucel-T.
  • Main analysis: narrative synthesis organized around biases and reproducibility, nanomedicine-specific issues, cell-therapy-specific issues, replication and translational successes, and recommendations summarized in tables and a checklist.

Limitations

  • The paper states that there is no systematic empirical study assessing what proportion of preclinical nanomedicine or cell therapy studies have had replication attempts, how many failed, what proportion reached human trials, or what proportion succeeded/licensed.
  • The paper states that no systematic assessment has been carried out on whether successful translational efforts were linked to particularly pristine preclinical studies, and it cautions that expectations need to be tempered because not all checklist items will be equally easy to tackle.
  • Evident from the text: as a Perspective, it relies on selected examples and cited literature rather than a systematic review or primary data collection, so the recommendations are expert synthesis rather than empirically tested interventions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41551-018-0314-y

Cites

125 of 125 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 93 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    The clinical translation of promising fundamental discoveries and preclinical approaches in nanomedicine and cell therapy has been challenging and inefficient.

    “The clinical translation of promising products, technologies and interventions from the disciplines of nanomedicine and cell therapy has been slow and inefficient.”
    • Also stated in: § Abstract · plain
      “the clinical translation of promising fundamental discoveries and preclinical approaches in nanomedicine and cell therapy, which hold great promise for the design of future medical interventions and for the improvement of current medical technologies, has been challenging and inefficient1,2.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [1] Bowen, A. & Casadevall, A. Increasing disparities between resource inputs and outcomes, as measured by certain health deliverables, in biomedical research. Proc. Natl Acad. Sci. USA 112, 11335–11340 (2015). source

      Not checked in the free run.

    • cites [2] Contopoulos-Ioannidis, D. G., Ntzani, E. & Ioannidis, J. P. Translation of highly promising basic science research into clinical applications. Am. J. Med. 114, 477–484 (2003). source

      Not checked in the free run.

  • Premisecitation

    Reproducibility checks in preclinical biomedical research have yielded very low rates of successful replication.

    “Allowing for these caveats, reproducibility checks in preclinical biomedical research have yielded very low rates of successful replication.”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [9] Prinz, F., Schlange, T. & Asadullah, K. Believe it or not: how much can we rely on published data on potential drug targets? Nat. Rev. Drug Discov. 10, 712 (2011). source

      Not checked in the free run.

    • cites [10] Begley, C. G. & Ellis, L. M. Drug development: raise standards for preclinical cancer research. Nature 483, 531–533 (2012). source

      Not checked in the free run.

    • cites [8] Ioannidis, J. P. Acknowledging and overcoming nonreproducibility in basic and preclinical research. JAMA 317, 1019–1020 (2017). source

      Not checked in the free run.

    • cites [11] Nosek, B. A. & Errington, T. M. Making sense of replications. eLife 6, e23383 (2017). source

      Not checked in the free run.

  • Interpretationcitation

    Lack of reproducibility may herald genuine heterogeneity in biological and experimental systems that is poorly controlled or not well understood.

    “It may herald genuine heterogeneity in biological and experimental systems5,6 that is poorly controlled or not well understood.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [5] Lithgow, G. J., Driscoll, M. & Phillips, P. A long journey to reproducible results. Nature 548, 387–388 (2017). source

      Not checked in the free run.

    • cites [6] Bissell, M. Reproducibility: the risks of the replication drive. Nature 503, 333–334 (2013). source

      Not checked in the free run.

  • Premisecitation

    Randomization and blinding of investigators who analyse the results of animal experiments are used in less than 20% of studies.

    “Some pivotal aspects of study design, such as randomization and the blinding of investigators who analyse the results of animal experiments, are used in less than 20% of studies16–18”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [16] Hess, K. R. Statistical design considerations in animal studies published recently in Cancer Research. Cancer Res. 71, 625 (2011). source

      Not checked in the free run.

    • cites [17] Kilkenny, C. et al. Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS ONE 4, e7824 (2009). source

      Not checked in the free run.

    • cites [18] Steward, O., Popovich, P. G., Dietrich, W. D. & Kleitman, N. Replication and reproducibility in spinal cord injury research. Exp. Neurol. 233, 597–605 (2012). source

      Not checked in the free run.

  • Premisecitation

    Only 11% (6 of 53) of oncology drug-target studies published by academic investigators could be reproduced.

    “Similarly, only 11% (6 of 53) of oncology drug-target studies published by academic investigators could be reproduced10.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Begley, C. G. & Ellis, L. M. Drug development: raise standards for preclinical cancer research. Nature 483, 531–533 (2012). source

      Not checked in the free run.

  • May combine two claims. 35
  • Classified as probably not a substantial claim. 21
  • May combine two claims.

    Interpretationcitation

    Much of the difficulty to achieve desirable clinical translation may stem from lack of reproducibility and from biases in the early stages of the translational pipeline.

    “Much of the difficulty to achieve the desirable clinical translation may stem from lack of reproducibility and from biases in the early stages of the translational pipeline3,4.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [3] Baker, M. 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454 (2016). source

      Not checked in the free run.

    • cites [4] Allison, D. B., Brown, A. W., George, B. J. & Kaiser, K. A. Reproducibility: a tragedy of errors. Nature 530, 27–29 (2016). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Study registration ahead of study execution may help to improve transparency and to differentiate between unspecified post-hoc explorations and pre-specified design and analysis features.

    “Study registration ahead of study execution may help to improve transparency and to differentiate between unspecified post-hoc explorations and pre-specified design and analysis features.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    Diversity in the physical, chemical and biological properties of material substrates used for nanomedicines highlights the potential for significant heterogeneity and for bias, making the generalization of observed phenomena difficult, and limiting the translational potential of discoveries.

    “However, the diversity in the physical, chemical and biological properties of material substrates used for nanomedicines also highlights the potential for significant heterogeneity and for bias, making the generalization of observed phenomena difficult, and limiting the translational potential of discoveries47,48.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [47] Shi, J., Kantoff, P. W., Wooster, R. & Farokhzad, O. C. Cancer nanomedicine: progress, challenges and opportunities. Nat. Rev. Cancer 17, 20–37 (2017). source

      Not checked in the free run.

    • cites [48] Jiang, W. et al. Designing nanomedicine for immuno-oncology. Nat. Biomed. Eng. 1, 0029 (2017). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The versatility of nanomaterial synthesis enables researchers to readily develop nanomedicines with unique physicochemical properties tailored to specific biological applications, but it also makes the generalizability of nanomedicine studies difficult.

    “Although the versatility of nanomaterial synthesis enables researchers to readily develop nanomedicines with unique physicochemical properties tailored to specific biological applications, it also makes the generalizability of nanomedicine studies difficult.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Premisecitation

    Although most animal experiments in the context of neurological diseases find significant results, very few of these materialize in human applications.

    “Although most animal experiments in the context of neurological diseases find significant results, very few of these materialize in human applications15.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [15] Tsilidis, K. K. et al. Evaluation of excess significance bias in animal studies of neurological diseases. PLoS Biol. 11, e1001609 (2013). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    The use of questionable and detrimental research practices is probably caused by the lack of sufficient methodological and statistical expertise, and can be affected by the expectation and pressure to deliver significant results.

    “The use of questionable and detrimental research practices is probably caused by the lack of sufficient methodological and statistical expertise, and can be affected by the expectation and pressure to deliver significant results.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Results that show consistent promise across several laboratories may have better chances of success in further translational efforts.

    “Results that show consistent promise across several laboratories may have better chances of success in further translational efforts.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Tens of billions of US dollars are wasted annually owing to lack of reproducibility in preclinical research.

    “Various calculations estimate that tens of billions of US dollars are wasted annually owing to lack of reproducibility in preclinical research25,26.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [25] Freedman, L. P., Cockburn, I. M. & Simcoe, T. S. The economics of reproducibility in preclinical research. PLoS Biol. 13, e1002165 (2015). source

      Not checked in the free run.

    • cites [26] Ioannidis, J. P. A. et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet 383, 166–175 (2014). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Collaboration between investigators can increase both the credibility and trust of the research and its translational potential.

    “Collaboration between investigators can increase both the credibility and trust of the research and its translational potential.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationcitation

    Collaborative multi-site studies may allow for the optimization of power and for the evaluation of the consistency of results across multiple laboratories.

    “collaborative multi-site studies29 may allow for the optimization of power and for the evaluation of the consistency of results across multiple laboratories.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [29] Dirnagl, U. et al. A concerted appeal for international cooperation in preclinical stroke research. Stroke 44, 1754–1760 (2013). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Most preclinical studies are too small, which increases the chances of false-negative and false-positive results and of exaggerated conclusions.

    “Several evaluations have shown that most preclinical studies are too small, which increases the chances of false-negative and false-positive results and of exaggerated conclusions.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [12] Begley, C. G. & Ioannidis, J. P. Reproducibility in science: improving the standard for basic and preclinical research. Circ. Res. 116, 116–126 (2015). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    These deficiencies are not the result of fraud, which is rare in science at large, but reflect questionable and detrimental research practices, which most researchers unfortunately adopt.

    “These deficiencies are not the result of fraud, which is rare in science at large, but reflect questionable and detrimental research practices, which most researchers unfortunately adopt24.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [24] Fanelli, D. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS ONE 4, e5738 (2009). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Experimental variations along each developmental stage of a complex multifunctional nanomedicine can contribute to potential biases and reduce reproducibility, and biases may be further amplified during successive steps.

    “From the initial step of material synthesis to the preclinical testing of candidate systems in vivo, experimental variations along each developmental stage can contribute to potential biases and reduce reproducibility. Biases may be further amplified during successive steps.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Comparing notes when results from different laboratories diverge can reveal potential sources of heterogeneity and lead to the design of future experiments that account for such heterogeneity.

    “comparing notes when results from different laboratories diverge can reveal potential sources of heterogeneity and lead to the design of future experiments that account for such heterogeneity.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    A culture of open sharing is likely to accelerate the translation of successful lines of investigation, may allow picking errors and sources of heterogeneity and bias earlier in the process, and may help abandon non-working options faster.

    “Therefore, a culture of open sharing is likely to accelerate the translation of successful lines of investigation, may allow picking errors and sources of heterogeneity and bias earlier in the process, and may help abandon non-working options faster.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    Cell therapies may succeed in an animal model but the same cells may provide little benefit or no benefit in clinical studies (and vice versa).

    “Indeed, cell therapies may succeed in an animal model but the same cells may provide little benefit or no benefit in clinical studies (and vice versa).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Premisecitation

    Studies using these cells in non-human primates have not resolved concerns because evidence for behavioural change was ambivalent (some positive change at low doses but not at high doses) and post-mortem evidence of co-localization of transplanted-cell markers with dopamine did not exclude cell fusion.

    “Studies using these cells in non-human primates80 haven’t resolved the concerns because the evidence for any behavioural change in the monkeys was ambivalent (some positive change at low doses but not at high doses) and because post-mortem evidence of the co-localization of transplanted-cell markers with dopamine didn’t exclude cell fusion, which is not uncommon81.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [80] Gonzalez, R. et al. Neural stem cells from human parthenogenetic stem cells engraft and promote recovery in a nonhuman primate model of Parkinson’s disease. Cell Transplant. 25, 1945–1966 (2016). source

      Not checked in the free run.

    • cites [81] Wagers, A. J. & Weissman, I. L. Plasticity of adult stem cells. Cell 116, 639–648 (2004). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    A comprehensive study of human iPSC-derived dopamine neuron progenitor cells expressing specific markers of the neural floor plate (CORIN+; FOXA2+; TUJ1+; PAX6−) transplanted into a monkey model of Parkinson's disease showed stronger evidence of behavioural benefits, cellular integration of mature dopamine neurons within the putamen, and extension of dense neurites into the host striatum.

    “A comprehensive study of human iPSC-derived dopamine neuron progenitor cells expressing specific markers of the neural floor plate (CORIN+; FOXA2+; TUJ1+; PAX6−) and transplanted into a monkey model of Parkinson’s disease showed stronger evidence of behavioural benefits, of cellular integration of mature dopamine neurons within the putamen, and of extension of their dense neurites into the host st”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [82] Kikuchi, T. et al. Human iPS cell-derived dopaminergic neurons function in a primate Parkinson’s disease model. Nature 548, 592–596 (2017). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    For autologous cell therapies, probably more heterogeneity is tolerated because the patient receives their own cells and therefore their body is less likely to mount an immune response against the transplanted cells.

    “For autologous cell therapies, probably more heterogeneity is tolerated because the patient receives their own cells and therefore their body is less likely to mount an immune response against the transplanted cells.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    Transplanted neural stem cells of a PAX6+ dorsal neural fate are unlikely to produce dopaminergic neurons; PAX6− ventral midbrain neural floor plate precursor cells usually produce the dopaminergic neurons necessary for correcting Parkinson's disease.

    “these transplanted neural stem cells are of a fate (PAX6+ dorsal neural) that is unlikely to produce dopaminergic neurons (instead, the PAX6− ventral midbrain neural floor plate precursor cells usually produce the dopaminergic neurons that are necessary for correcting Parkinson’s disease79).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [79] Kriks, S. et al. Floor plate-derived dopamine neurons from hESCs efficiently engraft in animal models of PD. Nature 480, 547–551 (2011). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    MSCs derived from bone marrow, fat tissue or umbilical cord tissue, or differentiated from pluripotent stem cells, are all phenotypically different but share common marker antigens.

    “An example is the use of MSCs derived from the bone marrow, fat tissue or umbilical cord tissue, or differentiated from pluripotent stem cells. They are all phenotypically different but share common marker antigens73.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [73] Bianco, P. “Mesenchymal” stem cells. Annu. Rev. Cell Dev. Biol. 30, 677–704 (2014). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    The phagocytic capacity of human THP-1 cells may differ significantly from that of murine bone marrow-derived macrophages or of tissue-specific phagocytes such as hepatic Kupffer cells or alveolar macrophages, thus resulting in potential conflicting results under different experimental conditions.

    “The phagocytic capacity of these human cells may differ significantly from that of murine bone marrow-derived macrophages or of tissue-specific phagocytes such as hepatic Kupffer cells or alveolar macrophages, thus resulting in potential conflicting results under different experimental conditions.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Most of the transplanted cells expressed FOXA2 and a third expressed tyrosine hydroxylase consistent with mature functional dopamine neurons.

    “Most of the transplanted cells expressed FOXA2 and a third expressed tyrosine hydroxylase consistent with mature functional dopamine neurons.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [82] Kikuchi, T. et al. Human iPS cell-derived dopaminergic neurons function in a primate Parkinson’s disease model. Nature 548, 592–596 (2017). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The firefly luciferase protein may be immunogenic and may result in murine T-cell-mediated responses, leading to variations in tumour growth.

    “studies involving immune nanomedicines need to account for the potential immunogenicity of the firefly luciferase protein, which may result in murine T-cell-mediated responses, leading to variations in tumour growth67,68.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [67] Limberis, M. P., Bell, C. L. & Wilson, J. M. Identification of the murine firefly luciferase-specific CD8 T-cell epitopes. Gene Therapy 16, 441–447 (2009). source

      Not checked in the free run.

    • cites [68] Baklaushev, V. P. et al. Luciferase expression allows bioluminescence imaging but imposes limitations on the orthotopic mouse (4T1) model of breast cancer. Sci. Rep. 7, 7715 (2017). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    These technological advancements offer protocols for nanomaterial synthesis that are compliant with good manufacturing practice (GMP), thus greatly reducing inter-batch variability and potentially improving the reproducibility of nanomedicine studies.

    “These technological advancements now offer protocols for nanomaterial synthesis that are compliant with good manufacturing practice (GMP), thus greatly reducing inter-batch variability and potentially improving the reproducibility of nanomedicine studies.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    An improved understanding of complex biological mechanisms will help the reproducibility of the results.

    “Critically, an improved understanding of complex biological mechanisms will help the reproducibility of the results.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    With large datasets, it is difficult to design preclinical studies without statistical input to address the influence of multiple factors and their interactions on signalling-pathway analyses.

    “With large datasets, it is difficult to design preclinical studies without statistical input to address the influence of multiple factors and their interactions on signalling-pathway analyses.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    The lack of reproducibility of CAR-T cells may indicate a mismatch of the CAR-T antibody design and tumour recognition or access, or may result from yet unknown factors.

    “The lack of reproducibility may indicate a mismatch of the CAR-T antibody design and tumour recognition or access, or may result from yet unknown factors.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    Mice reconstituted with a functional human immune system (humanized mice) can be used for studying the therapeutic effects of nanomedicines against autoimmune diseases such as diabetes, or to harbour patient-derived tumours for preclinical testing of cancer immune nanomedicines.

    “Similarly, mice reconstituted with a functional human immune system (humanized mice) can be used for studying the therapeutic effects of nanomedicines against autoimmune diseases such as diabetes, or to harbour patient-derived tumours for preclinical testing of cancer immune nanomedicines95.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [95] Rongvaux, A. et al. Development and function of human innate immune cells in a humanized mouse model. Nat. Biotechnol. 32, 364–372 (2014). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    These are robust data that encourage widespread clinical adoption.

    “These are robust data that encourage widespread clinical adoption.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    The recent approval by the FDA of CAR-T cell therapy for patients 25 years or younger is a product that is effective for B-cell (CD19) acute lymphoblastic leukaemia (~80% clearance rate of cancers), but is expensive and has dangerous side effects (cytokine release syndrome and neurological symptoms).

    “The recent approval by the FDA of CAR-T cell therapy for patients 25 years or younger is a product that is effective for B-cell (CD19) acute lymphoblastic leukaemia (~80% clearance rate of cancers)104, but is expensive and has dangerous side effects (cytokine release syndrome and neurological symptoms).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [104] Wei, G., Wang, J., Huang, H. & Zhao, Y. Novel immunotherapies for adult patients with B-lineage acute lymphoblastic leukemia. J. Hemat. Oncol. 10, 150 (2017). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Data sharing between major pharmaceutical companies can accelerate clinical drug development.

    “Data sharing between major pharmaceutical companies can accelerate clinical drug development110”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [110] Mayo-Wilson, E., Doshi, P. & Dickersin, K. Are manufacturers sharing data as promised?. Br. Med. J. 351, h4169 (2015). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Only 8 cell therapies have received approval to date by the FDA, and another 4 have received approval from the EMA, despite extensive preclinical work and many early clinical trials on cell therapies.

    “Only 8 such therapies have received approval to date by the FDA, and another 4 have received approval from the European Medicines Agency (EMA), despite extensive preclinical work and many early clinical trials on cell therapies89.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [89] Trounson, A. & McDonald, C. Stem cell therapies in clinical trials: progress and challenges. Cell Stem Cell 17, 11–22 (2015). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    A comprehensive account of cancer-nanomedicine products approved by the FDA lists 14 products and another 35 that are in different phases of clinical trials for cancer therapy.

    “In fact, a comprehensive account of cancer-nanomedicine products approved by the United States Food and Drug Administration (FDA) lists 14 products and another 35 that are in different phases of clinical trials for cancer therapy50.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [50] von Roemeling, C. A., Jiang, W., Chan, C. K., Weissman, I. L. & Kim, B. Y. S. Breaking down the barriers to precision cancer nanomedicine. Trends Biotechnol. 35, 159–171 (2017). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Clinical benefit in patients occurred up to 9 months after cell transplantation, but was lost after 12 months perhaps owing to a delayed immune response that rejected the allogeneic transplant.

    “Clinical benefit in patients occurred up to 9 months after cell transplantation, but was lost after 12 months perhaps owing to a delayed immune response that rejected the allogeneic transplant.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [87] Temple, S. & Studder, L. Lessons learned from pioneering neural stem cell studies. Stem Cell Rep. 8, 191–193 (2017). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    CAR-T cells with the same antibody have been less successful and less reproducible for other B-cell tumours (such as chronic lymphoblastic leukaemia) and for other antibodies and solid tumours.

    “CAR-T cells with the same antibody have been less successful and less reproducible for other B-cell tumours (such as chronic lymphoblastic leukaemia) and for other antibodies and solid tumours107.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [107] Fesnak, A., June, C. H. & Levine, B. L. Engineered T cells: the promise and challenges of cancer immunotherapy. Nat. Rev. Cancer 16, 566–681 (2016). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    There are numerous publications about progress in data sharing for immune cell therapies.

    “(although there are numerous publications about progress in data sharing for immune cell therapies)111,112”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [111] Guinney, J. et al. Prediction of overall survival for patients with metastatic castration-resistant prostate cancer: development of a prognostic model through a crowdsourced challenge with open clinical trial data. Lancet Oncol. 18, 132–142 (2017). source

      Not checked in the free run.

    • cites [112] Corrie, B. D. et al. iReceptor: a platform for querying and analysing antibody/B-cell and T-cell receptor repertoire data across federated repositories. Immunol. Rev. 284, 24–41 (2018). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    At present, there is little consensus on processing and manufacturing standards, on cryopreservation and on the degree of heterogeneity of the cell population that is acceptable.

    “At present, there is little consensus on processing and manufacturing standards100, on cryopreservation101 and on the degree of heterogeneity of the cell population that is acceptable102.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [100] Dropulic, B. Reference standards for gene and cell therapy products. Mol. Therapy 25, 1259–1260 (2017). source

      Not checked in the free run.

    • cites [101] Stacey, G. N. et al. Preservation and stability of cell therapy products: recommendations from an expert workshop. Regen. Med. 12, 553–564 (2017). source

      Not checked in the free run.

    • cites [102] Williams, D. J. et al. Comparability: manufacturing, characterization and controls, report of a UK regenerative medicine platform pluripotent stem cell platform workshop, Trinity Hall, Cambridge, 14-15 September 2015. Regen. Med. 11, 483–492 (2016). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    A cancer-nanomedicine repository spearheaded by the University of Toronto enables researchers to share their own experimental data, and has allowed for the determination of the delivery efficiency of specific nanomaterials into tumours.

    “A cancer-nanomedicine repository (http://inbs.med.utoronto.ca/cgi-bin/Repository.cgi) spearheaded by the University of Toronto enables researchers to share their own experimental data, and has allowed for the determination of the delivery efficiency of specific nanomaterials into tumours46.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [46] Wilhelm, S. et al. Analysis of nanoparticle delivery to tumours. Nat. Rev. Mater. 1, 16014 (2016). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Standards that have failed to evolve can underpin clinical reproducibility.

    “this suggests that standards that have failed to evolve can underpin clinical reproducibility.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The commercialization process does not favour data sharing because of intellectual-property issues.
  • May combine two claims.

    Premisecitation

    There has been some considerable variation in patient response and some serious neurological side-effects of CD19 CAR-T treatments that could not have been predicted from animal studies (rodents).

    “The high efficacy has been critical for the rapid adoption and registration of these CD19 CAR-T cells, but there has been some considerable variation in patient response and some serious neurological side-effects of the treatments that could not have been predicted from animal studies (rodents)105.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [105] Brudno, J. N. & Kochenderfer, J. N. Toxicities of chimeric antigen receptor T cells: recognition and management. Blood 127, 3321–3330 (2016). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    One year after autologous limbal cell grafting, the cells were 80% successful or partially successful; three years later 68% were judged to be successful.

    “One year after autologous cell grafting, the cells were 80% successful or partially successful; three years later 68% were judged to be successful103.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [103] Rama, P., Ferrari, G. & Pellegrini, G. Cultivated limbal epithelial transplantation. Curr. Opin. Ophthalmol. 28, 387–389 (2017). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Cell selection based on function potential is essential for the repeatability of outcomes in vitro and in vivo.

    “Cell selection based on function potential is essential for the repeatability of outcomes in vitro and in vivo.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    The Foundation for the Accreditation of Cellular Therapies (FACT) defined common standards for cellular therapy that include quality management for clinical processes and documentation; still, it does not yet address many of the quality issues affecting reproducibility.

    “The Foundation for the Accreditation of Cellular Therapies (FACT; http://www.factwebsite.org) defined common standards for cellular therapy that include quality management for clinical processes and documentation; still, it doesn’t yet address many of the quality issues affecting reproducibility.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites http://www.factwebsite.org

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    'Clustergrams' enable the visualization of biomarker and transcriptional data patterns.

    “‘Clustergrams’ (graphs for visualizing hierarchical and non-hierarchical cluster analyses) enable the visualization of biomarker and transcriptional data patterns70.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [70] Rennert, R. C. et al. High-resolution microfluidic single-cell transcriptional profiling reveals clinically relevant subtypes among human stem cell populations commonly utilized in cell-based therapies. Front. Neurology 7, 41 (2016).

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    As nanomedicine design becomes increasingly complex to accomplish a wider range of functional specifications, new manufacturing processes will be required to ensure successful transition for clinical testing.

    “However, as nanomedicine design becomes increasingly complex to accomplish a wider range of functional specifications, new manufacturing processes will be required to ensure successful transition for clinical testing.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Such curation requires a major investment, typically attainable only through cooperative consortia.

    “Such curation requires a major investment, typically attainable only through cooperative consortia.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    These methods enhance the repeatability of cell-therapy experiments and should be considered in experimental study design.

    “These methods enhance the repeatability of cell-therapy experiments and should be considered in experimental study design.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Common threads in nanomedicine and cell therapy are the optimization of technical facets, the relevance of the models used, data transparency and sharing, and the optimization of the study design and statistical methods.

    “Common threads are the optimization of technical facets, the relevance of the models used, data transparency and sharing, and the optimization of the study design and statistical methods.”
    Quote in the paperSubstantial claim
49 of 49
  • The paper relies on this without support.

    For a product, technology or intervention to warrant clinical trials, there must be sufficient preclinical evidence of safety and efficacy.

    “For a product, technology or intervention to warrant clinical trials, there must be sufficient preclinical evidence of safety and efficacy.”
  • The paper relies on this without support.

    Biases in preclinical research are identifiable and correctable preemptively.

    “It may also point to the presence of biases that are identifiable and correctable preemptively.”
  • The paper relies on this without support.

    Current methods for nanomaterial synthesis depend predominantly on bulk solution chemistry, which limits the production of highly monodisperse and homogenous nanomaterials on a large scale.

    “However, current methods for nanomaterial synthesis depend predominantly on bulk solution chemistry, which limits the production of highly monodisperse and homogenous nanomaterials on a large scale.”
  • The paper relies on this without support.

    A significant degree of experimental heterogeneity exists in the nanomedicine literature, particularly for cancer-nanomedicine studies, owing to the wide range of tumour models used.

    “A significant degree of experimental heterogeneity exists in the nanomedicine literature, in particular for cancer-nanomedicine studies, owing to the wide range of tumour models used.”
  • The paper relies on this without support.

    Many reported cancer-nanomedicine studies rely on human-xenograft tumours derived from long-term cultured cell lines implanted in immune-compromised mice.

    “A large amount of reported cancer-nanomedicine studies rely on the use of human-xenograft tumours, derived from long-term cultured cell lines that are implanted in immune-compromised mice.”
  • The paper relies on this without support.

    Variations in site of tumour inoculation, number of passages of the cell line, and use of extracellular-matrix materials to assist implantation can affect the biological properties of the tumour and result in different experimental outcomes.

    “Variations in the site of tumour inoculation (orthotopic versus ectopic sites), number of passages of the cell line and whether extracellular-matrix materials were used to assist tumour cell implantation, can all affect the biological properties of the tumour and result in different experimental outcomes.”
  • The paper relies on this without support.

    Nanomaterials possess a much higher affinity for interacting with immune cells than do conventional systemic therapies.

    “Since nanomaterials possess a much higher affinity for interacting with immune cells than do conventional systemic therapies”
  • The paper relies on this without support.

    Patient-derived xenograft (PDX) tumours closely resemble the disease in patients.

    “the use of patient-derived xenograft (PDX) tumours is also gaining increased interest, as these models closely resemble the disease in patients.”
  • The paper relies on this without support.

    Establishing a PDX model is time consuming and expensive, and requires rigorous molecular characterizations.

    “Establishing a PDX model is time consuming and expensive, and requires rigorous molecular characterizations.”
  • The paper relies on this without support.

    For tumour cells with low luciferase-transfection efficiency, there is discordance between the actual tumour size and the bioluminescence-intensity profile in vivo.

    “This is especially the case for tumour cells with low luciferase-transfection efficiency, which results in discordance between the actual tumour size and the bioluminescence-intensity profile in vivo.”
  • The paper relies on this without support.

    Populations of cells are rarely synchronized in their mitotic cell cycle and often include a variety of cell types or subtypes that may be in various stages of maturity or differentiation.

    “Populations of cells are rarely synchronized in their mitotic cell cycle and often include a variety of cell types or subtypes that may be in various stages of maturity or differentiation.”
  • The paper relies on this without support.

    There is no regulatory guidance about the degree of acceptable heterogeneity of cells used in cell therapies.

    “There is no regulatory guidance about the degree of acceptable heterogeneity of cells used in cell therapies.”
  • The paper relies on this without support.

    MSCs are not clonally derived and there is usually no attempt to control for heterogeneity in gene expression or in functionality, except when carrying out cell-surface-marker screens or when measuring the cells’ adherence to plastic surfaces or other general properties.

    “MSCs are not clonally derived and there is usually no attempt to control for heterogeneity in gene expression or in functionality, except when carrying out cell-surface-marker screens or when measuring the cells’ adherence to plastic surfaces or other general properties.”
  • The paper relies on this without support.

    Cultured (or expanded) cells can develop mutations and other genetic errors such as genomic deletions or rearrangements.

    “Cultured (or expanded) cells can develop mutations and other genetic errors such as genomic deletions or rearrangements.”
  • The paper relies on this without support.

    Mutations may also affect genes that are critical for the specific functions of the transplanted cells.

    “Mutations may also affect genes that are critical for the specific functions of the transplanted cells.”
  • The paper relies on this without support.

    Pluripotent stem cells may form teratomas in vivo.

    “Pluripotent stem cells may form teratomas in vivo”
  • The paper relies on this without support.

    The stage of maturity of cells produced via directed differentiation of stem cells may add to cellular heterogeneity and can be critical to the outcome of therapy.

    “The stage of maturity of cells produced via directed differentiation of stem cells may also add to cellular heterogeneity, and can be critical to the outcome of therapy.”
  • The paper relies on this without support.

    In patients, CAR-T cells function for about 3–8 weeks.

    “In patients, CAR-T cells function for about 3–8 weeks.”
  • The paper relies on this without support.

    To our knowledge there is no systematic empirical study that has assessed what proportion of preclinical studies in nanomedicine or cell therapy have had attempts at replication, and how many of those have failed.

    “To our knowledge there is no systematic empirical study that has assessed what proportion of preclinical studies in nanomedicine or cell therapy have had attempts at replication, and how many of those have failed.”
  • The paper relies on this without support.

    There is no systematic assessment of what proportion of the initial discoveries in nanomedicine and cell therapy have reached the stage of human trials and what proportion has been successful (licensed).

    “Similarly, there is no systematic assessment of what proportion of the initial discoveries in these fields have reached the stage of human trials and what proportion has been successful (that is, licensed).”
  • The paper relies on this without support.

    Multiple companies have already produced nanomedicine products that have resulted in clinical trials, providing further evidence that tackling the issues of GMP is feasible, at least for simple nanomedicine formulations such as PEGylated-proteins or liposomes.

    “Multiple companies have already produced nanomedicine products that have resulted in clinical trials, providing further evidence that tackling the issues of GMP is feasible, at least for simple nanomedicine formulations such as PEGylated-proteins or liposomes”
  • The paper relies on this without support.

    For cancer-nanomedicine studies, xenograft human tumour models in immune-deficient mice are beginning to be replaced by PDX and genetically engineered mouse models to study nanomedicine actions in more clinically relevant settings.

    “For cancer-nanomedicine studies, xenograft human tumour models in immune-deficient mice are beginning to be replaced by PDX and genetically engineered mouse models to study nanomedicine actions in more clinically relevant settings.”
  • The paper relies on this without support.

    An often-overlooked aspect in nanomedicine studies is the robustness of the statistical design of the experiments and of the analyses of the data.

    “An often-overlooked aspect in nanomedicine studies is the robustness of the statistical design of the experiments and of the analyses of the data.”
  • The paper relies on this without support.

    Data were repeatable and consistently high efficacy in vitro and in animal models was obtained for limbal cells.

    “Data were repeatable and consistently high efficacy in vitro and in animal models was obtained.”
  • The paper relies on this without support.

    Cell products tend to evolve from laboratories and be adopted by companies, with little reference to other similar or different cell products.

    “Cell products tend to evolve from laboratories and be adopted by companies, with little reference to other similar or different cell products.”
  • The paper relies on this without support.

    Mistakes in cell product development are often repeated.

    “Mistakes are often repeated.”
  • The paper relies on this without support.

    Empirical data showing that merged institutional experiences improved time-to-clinical-trial and the reproducibility of cell products have not been published.

    “These organizations claim to have improved time-to-clinical-trial and the reproducibility of cell products, but empirical data to this effect have not been published.”
  • The paper relies on this without support.

    There is no evidence of data sharing happening in cell therapies.

    “but there isn’t any evidence of this happening in cell therapies”
  • The paper relies on this without support.

    The commercialization process does not favour data sharing because of intellectual-property issues.

    “The commercialization process does not favour data sharing because of intellectual-property issues”
  • The paper relies on this without support.

    Many animal models do not represent the human condition sufficiently to be used for reliable predictions of efficacy in human patients.

    “Many animal models do not represent the human condition sufficiently to be used for reliable predictions of efficacy in human patients.”
  • The paper relies on this without support.

    The absence of a competent human immune system can make animal studies inadequate for genuine safety issues.

    “The absence of a competent human immune system can make animal studies inadequate for genuine safety issues.”
  • The paper relies on this without support.

    Treatments for autoimmune diseases, such as type 1 diabetes or multiple sclerosis, are difficult to model in a predictable way.

    “Treatments for autoimmune diseases, such as type 1 diabetes or multiple sclerosis, are difficult to model in a predictable way.”
  • The paper relies on this without support.

    In many cases, there is no reliable animal model because the mechanism of the disease is unknown or the condition cannot be replicated in animals.

    “In many cases, there is no reliable animal model because the mechanism of the disease is unknown or the condition cannot be replicated in animals.”
  • The paper relies on this without support.

    A systematic assessment of whether successful translational efforts in nanomedicine or cell therapy were linked to particularly pristine and meticulously executed preclinical studies has not been carried out.

    “To our knowledge, a systematic assessment of whether successful translational efforts in nanomedicine or cell therapy (or in other bench-discovery fields) were linked to particularly pristine and meticulously executed preclinical studies has not been carried out.”
  • The paper relies on this without support.

    The methods for optimal study designs and statistical analyses are already available and are simply underused or ignored.

    “since the methods are already available and they are simply underused or ignored”
35 of 35

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • QoS includes high data rates, extremely reliable and low latency communication (ERLLC), further-enhanced mobile broadband (FeMBB), ultra-massive machine-type communications (umMTC), long-distance and high-mobility communications (LDHMC) and extremely low-power communications (ELPC).

    “For instance, QoS includes high data rates, extremely reliable and low latency communication (ERLLC), further-enhanced mobile broadband (FeMBB), ultra-massive machine-type communications (umMTC), longdistance and high-mobility communications (LDHMC) and extremely low-power communications (ELPC) [18].”
  • Subtle changes in process or composition can adversely affect the complex composition of nanomedicines.

    “Subtle changes in process or composition can adversely affect the complex composition of nanomedicines 142,143 .”

    From Nanotechnology approaches for global infectious diseases (2021) · cites this paper as [143]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Ioannidis, J. P. A., Kim, B. Y. S., & Trounson, A.
Affiliations
  • Stanford Medicine (US)
  • Mayo Clinic (US)
  • Hudson Institute of Medical Research (AU)
Corresponding author
Ioannidis, John P. A.
Venue
Nature Biomedical Engineering
Published
2018-11-08
Volume
2
Issue
11
Pages
797-809
ISSN
2157-846X (ISSN-L)
OpenAlex
W2899457310
Cited by
175
References
125
Citations by year
2026: 23 · 2025: 17 · 2024: 27 · 2023: 13 · 2022: 22 · 2021: 27 · 2020: 33 · 2019: 12 · 2017: 1
FWCI
26.34
Type
journal article
Language
en
Open access
Locations
  • Nature Biomedical Engineering (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • Ontario Institute for Regenerative Medicine
  • National Institute of Neurological Disorders and Stroke (R01NS104315)
Topics
  • Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
Keywords
preclinical study design, nanomedicine, cell therapy, clinical translation, research bias, reproducibility, allocation bias, researcher bias, underpowered studies, statistical analysis, model heterogeneity, replication, research transparency, reporting practices, research incentives
MeSH terms
Trastuzumab, Ado-Trastuzumab Emtansine, Animals, Breast Neoplasms, Disease Models, Animal, Drug Evaluation, Preclinical, Female, Male, Maytansine, Prostatic Neoplasms, Research Design, Tissue Extracts, Reproducibility of Results, Nanomedicine, Cell- and Tissue-Based Therapy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review