Cited by 187Statistical Methods in Clinical TrialsComputational Drug Discovery MethodsOptimal Experimental Design Methods

Statistical determination of synergy based on Bliss definition of drugs independence

Open access
Demidenko, Eugene, Miller, Todd W.·2019·PLOS ONE

Abstract

Although synergy is a pillar of modern pharmacology, toxicology, and medicine, there is no consensus on its definition despite its nearly one hundred-year history. Moreover, methods for statistical determination of synergy that account for variation of response to treatment are underdeveloped and if exist are reduced to the traditional t-test, but do not comply with the normal distribution assumption. We offer statistical models for estimation of synergy using an established definition of Bliss drugs' independence. Although Bliss definition is well-known, it remains a theoretical concept and has never been applied for statistical determination of synergy with various forms of treatment outcome. We rigorously and consistently extend the Bliss definition to detect statistically significant synergy under various designs: (1) in vitro, when the outcome of a cell culture experiment with replicates is the proportion of surviving cells for a single dose or multiple doses, (2) dose-response methodology, (3) in vivo studies in organisms, when the outcome is a longitudinal measurement such as tumor volume, and (4) clinical studies, when the outcome of treatment is measured by survival. For each design, we developed a specific statistical model and demonstrated how to test for independence, synergy, and antagonism, and compute the associated p-value.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0224137

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing methods have limited power to detect synergy in animal studies.

    “Existing methods have limited power to detect synergy in animal studies.”

    From Statistical Assessment of Drug Synergy from <i>In Vivo</i> Combination Studies Using Mouse Tumor Models (2023) · cites this paper as [29]

    The Discussion (page unavailable) says existing statistical tests for synergy are scarce and applies the proposed method to animal studies. The single-dose section also says traditional whole-curve approaches may overlook synergy at particular doses. These passages support limitations in existing methods, but do not establish that those methods have limited statistical power specifically in animal studies. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • Existing methods use only TV data from a single day to calculate CI, and local CI values may fluctuate excessively such that they may indicate synergy and antagonism at different days.

    “or use only TV data from a single day to calculate CI (local CI values may fluctuate excessively such that they may indicate synergy and antagonism at different days)”

    From Statistical Assessment of Drug Synergy from <i>In Vivo</i> Combination Studies Using Mouse Tumor Models (2023) · cites this paper as [29]

    The “Tumor growth experiments in vivo” section (page unavailable) says several authors apply CI to tumor volume at a specific time point, supporting the single-day part of the claim. The Introduction says CI values can be affected by biological variation, but our copy does not say that local CI values fluctuate enough to indicate synergy on one day and antagonism on another. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • CI cannot be calculated after a day when most or all mice in the vehicle control group are euthanized because their tumors reach a size threshold faster than drug-treated tumors.

    “or that CI cannot be calculated after a day when most or all mice in the vehicle control group are euthanized because their tumors reach certain size threshold faster than drug-treated tumors.”

    From Statistical Assessment of Drug Synergy from <i>In Vivo</i> Combination Studies Using Mouse Tumor Models (2023) · cites this paper as [29]

    The Introduction’s “Advantages and limitations of Loewe additivity and combination index” section says CI is difficult to generalize to longitudinal animal experiments, including tumor growth. The “Tumor growth experiments in vivo” section says CI has been applied to tumor volume at a specific time point. Neither passage says that CI becomes incalculable after control mice are euthanized at a tumor-size threshold; no page numbers are available in our copy. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • Demidenko and colleagues defined Bliss independence using tumor growth rates of the four groups by assuming exponential tumor growth kinetics, and devised a Z-test using estimated GRs from linear mixed models on log-transformed TV data.

    “Specifically, Demidenko and Miller defined Bliss independence using tumor growth rates (GR) of the four groups by assuming exponential tumor growth kinetics, and devised a Z-test to assess whether Bliss synergy exists using estimated GRs obtained from fitting linear mixed models on log-transformed TV data (29).”

    From Statistical Assessment of Drug Synergy from <i>In Vivo</i> Combination Studies Using Mouse Tumor Models (2023) · cites this paper as [29]

    The “Tumor growth experiments in vivo” section defines Bliss independence using growth rates for four groups under an exponential-growth assumption. It also describes testing synergy with p-values calculated from slope estimates and variances obtained using linear mixed models on log-scale tumor-volume data. Our copy does not explicitly identify this procedure as a Z-test, so that specific characterization is not verified. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • Existing methods impose unrealistic assumptions for tumor growth and TV data.

    “These methods, however, have inherent limitations because they either impose unrealistic assumptions for tumor growth and TV data”

    From Statistical Assessment of Drug Synergy from <i>In Vivo</i> Combination Studies Using Mouse Tumor Models (2023) · cites this paper as [29]

    In “Tumor growth experiments in vivo” (page unavailable), the paper says its tumor-volume analysis assumes exponential growth or decay and notes that a U-shaped trajectory after short or one-time treatment requires a different model. This supports a limitation of a tumor-growth assumption, but the paper does not characterize existing methods generally as imposing unrealistic assumptions. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 54 · Cited by 187

Research works printed but not linked: 43

  1. [1]Greco WR, Bravo G, Parsons JC. The search for synergy: A critical review from response surface perspective. Pharmacological Reviews 1995; 47(2): 331– 385. 7568331
  2. [2]Foucquier J, Guedj M. Analysis of drug combinations: Current methodological landscape. Pharmacology Research & Perspectives 2015; 3: e00149 10.1002/prp2.149 26171228 PMC4492765 · DOI · Paper page
  3. [3]Loewe S. Die quantitation probleme der pharmakologie. Ergebnisse Physiologie 1928; 27(1): 47– 187. 10.1007/BF02322290 · DOI
  4. [4]Bliss CI. The toxicity of poisons applied jointly. Ann. Appl. Biol. 1939; 26(3): 585– 615. 10.1111/j.1744-7348.1939.tb06990.x · DOI · Paper page
  5. [5]Webb JL. Enzyme and Metabolic Inhibitors, vol. 1 New York and London: Academic Press; 1963.
  6. [6]Saul A, Fay MP. Human immunity and the design of multi-component, single target vaccines. PLoS ONE 2007; 2(9): e850 10.1371/journal.pone.0000850 17786221 PMC1952173 · DOI
  7. [7]Boik JC, Narasimhan B. An R package for assessing drug synergism/antagonism. Journal of Statistical Software 2010; 34(6): 1– 18. 10.18637/jss.v034.i06 · DOI
  8. [8]Kashif M, Andersson C, Mansoori M, Larsson R, Nygren P, Gustafsson MG. Bliss and Loewe interaction analyses of clinically relevant drug combinations in human colon cancer cell lines reveal complex patterns of synergy and antagonism. Oncotatget 2017; 8(61): 103952– 103967. PMC5732778 29262612
  9. [9]Flobak A, Vazquez M, Lægreid A, Valencia A. CImbinator: a web-based tool for drug synergy analysis in small- and large-scale datasets. Bioinformatics 2017; 33(15): 2410– 2412. 10.1093/bioinformatics/btx161 28444126 PMC5860113 · DOI
  10. [10]Chou TC, Talalay P. Analysis of combined drug effects: a new look at a very old problem. Trends in Pharmacological Sciences 1983; 4(11): 450– 454. 10.1016/0165-6147(83)90490-X · DOI
  11. [11]Chou TC. Theoretical basis, experimental design, and computerized simulation of synergism and antagonism in drug combination studies. Pharmacological Reviews 2006; 58(3): 621– 681. 10.1124/pr.58.3.10 16968952 · DOI · Paper page
  12. [12]Demidenko E. The p-value you can’t buy. American Statistician 2016; 70(1): 33– 38. 10.1080/00031305.2015.1069760 27226647 PMC4867863 · DOI
  13. [13]Demidenko E. Advanced Statistics with Applications in R. Hoboken, NJ: Wiley; 2019.
  14. [14]Chou TC, Talalay P. Generalized equations for the analysis of inhibitions of Michaelis-Menten and higher-order kinetic systems with two or more mutually exclusive and nonexclusive inhibitors. European Journal of Biochemistry 1981; 115(1): 207– 216. 10.1111/j.1432-1033.1981.tb06218.x 7227366 · DOI
  15. [16]Chou T. Synergy determination issues, Letter to the Editor. Journal of Virology 2002; 76(20): 10577– 10578. 10.1128/JVI.76.20.10577-10578.2002 12239339 PMC136565 · DOI
  16. [17]Tallarida RJ. The interaction index: a measure of drug synergism. Pain 2002; 98(1-2): 163– 168. 10.1016/s0304-3959(02)00041-6 12098628 · DOI
  17. [18]Chou T. Drug combination studies and their synergy quantification using the Chou-Talalay method. Cancer Research 2010; 70(2): 440– 446. 10.1158/0008-5472.CAN-09-1947 20068163 · DOI · Paper page
  18. [19]Geary N. Understanding synergy. American Journal of Physiology Endocrinology and Metabolism 2013; 304(3): E237– E253. 10.1152/ajpendo.00308.2012 23211518 · DOI
  19. [20]Demidenko E. Mixed Models: Theory and Applications with R. 2d ed Hoboken, NJ: Wiley; 2013.
  20. [21]Feller W. An Introduction to Probability Theory and Its Applications, 3d ed New York: Wiley; 1968.
  21. [22]Searle SR. Linear Models. 2d ed New York: Wiley; 1971.
  22. [24]Straetemans R, Bijnens L. Application and review of the separate ray model to investigate interaction effects. Frontiers in Bioscience 2010; 2: 266– 278. 10.2741/e89 20036877 · DOI
  23. [26]Wackerly DD, Mendenhall W, Scheaffer RL. Mathematical Statistics with Applications, 6th ed., Pacific Grove, CA: Duxbury Press; 2002.
  24. [27]Englehardt JD. Distributions of autocorrelated first-order kinetic outcomes: Illness Severity. PLoS ONE 2015; 10(6): e0129042 10.1371/journal.pone.0129042 26061263 PMC4465627 · DOI
  25. [28]Caudle R, Williams G. The misuse of analysis of variance to detect synergy in combination drug studies. Pain 1993; 55(3): 313– 317. 10.1016/0304-3959(93)90006-b 8121692 · DOI
  26. [29]Glaholt SP, Chen CY, Demidenko E, Bugge DM, Folt CL, Shaw JR. Adaptive iterative design (AID): A novel approach for evaluating the interactive effects of multiple stressors on aquatic organisms. Science of the Total Environment 2012; 432: 57– 64. 10.1016/j.scitotenv.2012.05.074 22717606 PMC3411930 · DOI
  27. [30]Demidenko E, Glaholt SP, Kyker-Snowman E, Shaw JR, Chen CY. Single toxin dose-response models revisited. Toxicology and Applied Pharmacology 2017; 314: 12– 23. 10.1016/j.taap.2016.11.002 27847315 PMC5183473 · DOI
  28. [31]Slinker BK. The statistics of synergism. Journal of Molecular Cell Cardiology 1998; 30(4): 723– 731. 10.1006/jmcc.1998.0655 9602421 · DOI
  29. [32]Hosford SR, Dillon LM, Bouley SJ, Rosati R, Yang W, Chen VS, Demidenko E, Morra RP, Miller TW. Combined inhibition of both p110 α and p110 β isoforms of phosphatidylinositol 3-kinase is required for sustained therapeutic effect in PTEN-deficient, ER + breast cancer. Clinical Cancer Research 2017; 23(11): 2795– 2805. 10.1158/1078-0432.CCR-15-2764 27903677 PMC5449270 · DOI
  30. [33]Cokol M, Chua HN, Tasan M, Mutlu B, Weinstein ZB, Suzuki Y, et al Systematic exploration of synergistic drug pairs. Molecular System Biology 2011; 7: 544 10.1038/msb.2011.71 PMC3261710 22068327 · DOI
  31. [35]Lederer S, Dijkstra TMH, Heskes T. Additive dose response models: Explicit formulation and the Loewe additivity consistency condition. Frontiers in Pharmacology 2018; 9: 31 10.3389/fphar.2018.00031 29467650 PMC5808155 · DOI
  32. [36]Carter WH, Gennings C, Staniswalis JG, Campbell ED, White KL. A statistical approach to the construction and analysis of isobolograms. Journal of the American College of Toxicology 1988; 7(7): 963– 973. 10.3109/10915818809014527 · DOI
  33. [38]Nelsen RB. An Introduction to Copulas. New York: Springer; 2016.
  34. [39]Fedorov VV, Leonov SL. Optimal Design for Nonlinear Response Models. Boca Raton, FL: CRC Press; 2014.
  35. [40]Ashford JR, Sowden RR. Multi-variate probit analysis. Biometrics 1970; 26(3): 535– 546. 10.2307/2529107 5480663 · DOI
  36. [41]Lesaffre E, Molenberghs G. Multivariate probit analysis: A neglected procedure in medical statistics. Statistics in Medicine 1991; 10(9): 1391– 1403. 10.1002/sim.4780100907 1925169 · DOI
  37. [42]Finney DJ. Probit Analysis, 2d ed Cambridge: Cambridge University Press; 1952.
  38. [43]Bijnsdorp I, Giovannetti E, Peters GP. Analysis of drug interactions. Methods Mol. Biol. 2011; 731: 421– 34. 10.1007/978-1-61779-080-5_34 21516426 · DOI
  39. [44]Verrier F, Nadas A, Gorny MK, Zolla-Pazner S. Additive effects characterize the interaction of antibodies involved in neutralization of the primary dualtropic human immunodeficient virus type 1 isolate 89.6. J. Virol. 2001; 75(19): 9177– 9186. 10.1128/JVI.75.19.9177-9186.2001 11533181 PMC114486 · DOI
  40. [45]Yu D, Kahen E, Cubitt CL, McGuire J, Kreahling J, Lee J, Altiok S, Lynch CC, Sullivan DM, Reed DR. Identification of synergistic, clinically achievable, combination therapies for osteosarcoma. Scientific Reports 2015; 5: 16991 10.1038/srep16991 26601688 PMC4658502 · DOI
  41. [46]Wang F, Dai W, Wang Y, Shen M, Chen K, Cheng P, et al The synergistic in vitro and in vivo antitumor effect of combination therapy with salinomycin and 5-fluorouracil against hepatocellular carcinoma. PLoS ONE 2014; 9(5): e97414 10.1371/journal.pone.0097414 24816638 PMC4016361 · DOI
  42. [48]Hampsch RA, Shee K, Bates D, Lewis LD, Désiré L, Leblond B, Demidenko E, et al Therapeutic sensitivity to Rac GTPase inhibition requires consequential suppression of mTORC1, AKT, and MEK signaling in breast cancer. Oncotarget 2017; 8(13): 21806– 21817. 10.18632/oncotarget.15586 28423521 PMC5400625 · DOI
  43. [50]Palmer AC, Sorger PK. Combination cancer therapy can confer benefit via patient-to-patient variability without drug additivity or synergy. Cell 2017; 171(7); 1878– 1691. 10.1016/j.cell.2017.11.009 PMC5741091 29245013 · DOI · Paper page

Authors
Demidenko, E., & Miller, T. W.
Affiliations
Dartmouth College (US)
Corresponding author
Demidenko, Eugene
Venue
PLOS ONE
Published
2019-11-25
Volume
14
Issue
11
Pages
e0224137
ISSN
1932-6203 (ISSN-L)
OpenAlex
W2989954637
Cited by
187
References
54
Citations by year
2026: 53 · 2025: 26 · 2024: 22 · 2023: 27 · 2022: 28 · 2021: 27 · 2020: 4
FWCI
11.51
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
Funders
  • National Institutes of Health
  • National Cancer Institute (U01CA196386, P30CA023108)
  • U.S. National Library of Medicine (R56LM012371)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
Keywords
drug synergy, Bliss independence, antagonism, dose-response relationship, cell survival, tumor volume, survival analysis, treatment response variability
MeSH terms
Animals, Antineoplastic Agents, Cell Survival, Clinical Trials as Topic, Disease Models, Animal, Dose-Response Relationship, Drug, Drug Antagonism, Drug Evaluation, Preclinical, Drug Synergism, Humans, Models, Biological, Neoplasms, Treatment Outcome, Cell Culture Techniques, Cell Line, Tumor, Mice, Kaplan-Meier Estimate, Intention to Treat Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review