Alternative drug sensitivity metrics improve preclinical cancer pharmacogenomics
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
VOLUME 35 NUMBER 6 JUNE 2017 NATURE BIOTECHNOLOGY were missed using IC50. An example of a false-negative association is provided by PTEN loss-of-function (LOF) mutations, which are known to mediate lapatinib (Tykerb) resistance in breast cancer cells10 but were not associated with differences in IC50 values (P = 0.12, rank-sum test) simply because PTEN mutant cell lines also had a median division time that was 22.5 h shorter than PTEN wild-type cell lines (P = 0.01), consistent with the known role of PTEN as a tumor suppressor. By contrast, the association between PTEN LOF mutations and lapatinib resistance was significant when GR50 values were used (P = 0.003; Fig. 1d). A false-positive example is provided by deletion of CDC73 (a component of an RNA polymerase II complex11), which was associated in ovarian cancer lines with a 1,000-fold increase in median IC50 for docetaxel (P = 0.002; Fig. 1e). CDC73-deleted cells grew more slowly that CDC73 expressers (57 h versus 35 h doubling time; P = 0.03), and when we corrected for this difference using GR50 values, the association between CDC73 deletion and docetaxel sensitivity disappeared (P = 0.13). We conclude that differences in cell division rate underlie false-positive and false-negative associations between genomic features and conventional drug response metrics. Efficacy is rarely explored as a metric of drug response, even though incomplete cytostasis and fractional cell killing result in persistence of viable cells12 that likely contributes to incomplete tumor regression and residual disease13. Across the gCSI data set, both potency and efficacy exhibited substantial variation, and even highly potent drugs could be inefficient at killing (Fig. 2a and Supplementary Fig. 2). For example, the mTOR inhibitor rapamycin was potent (GR50 < 0.1 μM) in 34% of cell lines but ineffective in blocking growth in these lines (GRmax = 0.34 ± 0.23, median and IQR). The converse was also true: the histone deacetylase (HDAC) inhibitor vorinostat (Zolinza) was highly cytotoxic in most lines (GRmax = –0.52 ± 0.48) To the Editor: Measuring drug response in cancer cell lines is essential for studying mechanisms of drug action and identifying genetic variants associated with sensitivity and resistance (preclinical pharmacogenomics). The conventional approach involves computing relative viability based on the ratio of cells in control and drug-treated cultures at the end of a fixed time period (Fig. 1a and Supplementary Fig. 1a). We have recently shown that relative viability and the response metrics derived from it, such as IC50 (drug concentration resulting in 50% relative viability) and area under the dose–response curve (AUC), are confounded by variation in cell proliferation rates and assay duration1, potentially explaining discrepancies among large-scale drug response data sets2,3. Here we show that use of relative viability as a measure of drug response results in falsepositive and false-negative pharmacogenomic associations. An alternative approach, based on computing normalized growth rate (GR) inhibition minimizes these artifactual associations by scoring drug sensitivity on a per-division basis1 (Fig. 1a). GR50, a measure of potency, is the concentration of drug that reduces cell proliferation rate by one-half; GRmax, a measure of efficacy, is the maximum effect of a drug at the highest tested concentration; and GRAOC combines these in an integrated ‘area over the curve’ value. The signs of GR values and GRmax relate directly to response phenotype: positive for partial growth inhibition, zero for complete cytostasis and negative for cell death. By contrast, Emax (relative viability at the highest tested concentration) values do not have this property as they are strongly confounded by proliferation rates1,4. Collecting GR values requires only modest changes in experimental approach5,6 and calculations can be performed online (http:// www.GRcalculator.org/). A recently released data set from the Genentech Cell Line Screening Initiative (gCSI)7 is unique among large-scale drug response data sets in reporting cell division times, which are required for GR calculations. Median cell line division times varied about twofold across tissue types and up to threefold within cell lines of a given type (note that division times also vary with culture conditions1,2). Lymphoid cancer lines had the shortest median division times at 29 h with an interquartile range (IQR) of 15 h, whereas breast cancer lines had the longest division times at 53 h (IQR = 16 h; Fig. 1b). All other things being equal, slower division rates increase IC50 and Emax values for simple arithmetic reasons (Fig. 1a)1. In the gCSI data set, both IC50 and Emax were in fact substantially higher for the slowest dividing cell types relative to the fastest (Spearman’s r > 0.61, P < 0.005). For drugs such as paclitaxel (Abraxane, Taxol), docetaxel (Taxotere), and gemcitabine (Gemzar), drug sensitivity also correlated with cell division times (r > 0.35, P < 5.6 × 10-13 for IC50; r > 0.4, P < 1.4 × 10-16 for Emax). On the basis of IC50 values, bone cancer cells were reportedly 60-fold more drug sensitive on average than breast cancer cells even though the drugs in the gCSI data set are primarily used to treat the latter disease. By contrast, we found that median GR50 and GRmax values are not significantly correlated across fastand slowgrowing tumor cell types (P > 0.20). Thus, the correlation between tumor type and median IC50 or Emax values arises from systematic variation in division time. Conventional drug sensitivity metrics are similarly confounded by genetic mutations (natural and engineered) that alter proliferation rates8,9 (Supplementary Fig. 1b–d), resulting in both bona fide and confounded pharmacogenomic associations. Overall, only 40% of 122 statistically significant associations (falsediscovery rate (FDR) < 0.15) between IC50 values and genomic alterations (gene mutations, deletions, or amplifications) in gCSI data were significant when using GR50 values (Fig. 1c). Conversely, about half of associations scored as significant by GR50 Alternative drug sensitivity metrics improve preclinical cancer pharmacogenomics CORRESPONDENCE
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Harnessing Connectivity in a Large-Scale Small-Molecule Sensitivity Dataset
Seashore-Ludlow, B., Rees, M. G., Cheah, J. H., Cokol, M., Price, E. V., Coletti, M. E., Jones, V., Bodycombe, N. E., Soule, C. K., Gould, J., et al. (2015) · Cancer Discovery · doi:10.1158/2159-8290.cd-15-0235 · Cited by 813
Growth rate inhibition metrics correct for confounders in measuring sensitivity to cancer drugs
Hafner, M., Niepel, M., Chung, M., & Sorger, P. K. (2016) · Nature Methods · doi:10.1038/nmeth.3853 · Cited by 681
Self-citationInconsistency in large pharmacogenomic studies
Haibe-Kains, B., El-Hachem, N., Birkbak, N. J., Jin, A. C., Beck, A. H., Aerts, H. J. W. L., & Quackenbush, J. (2013) · Nature · doi:10.1038/nature12831 · Cited by 499
Eichhorn, P. J. A., Gili, M., Scaltriti, M., Serra, V., Guzman, M., Nijkamp, W., Beijersbergen, R. L., Valero, V., Seoane, J., Bernards, R., et al. (2008) · Cancer Research · doi:10.1158/0008-5472.can-08-1740 · Cited by 499
Pharmacogenomic agreement between two cancer cell line data sets
The Cancer Cell Line Encyclopedia Consortium, & The Genomics of Drug Sensitivity in Cancer Consortium (2015) · Nature · doi:10.1038/nature15736 · Cited by 418
An unbiased metric of antiproliferative drug effect in vitro
Harris, L. A., Frick, P. L., Garbett, S. P., Hardeman, K. N., Paudel, B. B., Lopez, C. F., Quaranta, V., & Tyson, D. R. (2016) · Nature Methods · doi:10.1038/nmeth.3852 · Cited by 92
Cited by
PI3K inhibitors are finally coming of age
Vanhaesebroeck, B., Perry, M. W. D., Brown, J. R., André, F., & Okkenhaug, K. (2021) · Nature Reviews Drug Discovery · doi:10.1038/s41573-021-00209-1 · Cited by 476
Keenan, A. B., Jenkins, S. L., Jagodnik, K. M., Koplev, S., He, E., Torre, D., Wang, Z., Dohlman, A. B., Silverstein, M. C., Lachmann, A., et al. (2018) · Cell Systems · doi:10.1016/j.cels.2017.11.001 · Cited by 423
Self-citationHafner, M., Mills, C. E., Subramanian, K., Chen, C., Chung, M., Boswell, S. A., Everley, R. A., Liu, C., Walmsley, C. S., Juric, D., et al. (2019) · Cell Chemical Biology · doi:10.1016/j.chembiol.2019.05.005 · Cited by 250
Self-citationArtificial Intelligence and Biosensors in Healthcare and Its Clinical Relevance: A Review
(2023) · doi:10.1109/access.2023.3285596 · Cited by 205
PharmacoDB: an integrative database for mining in vitro anticancer drug screening studies
Smirnov, P., Kofia, V., Maru, A., Freeman, M., Ho, C., El-Hachem, N., Adam, G.-A., Ba-alawi, W., Safikhani, Z., & Haibe-Kains, B. (2018) · Nucleic Acids Research · doi:10.1093/nar/gkx911 · Cited by 155
Fröhlich, F., Kessler, T., Weindl, D., Shadrin, A., Schmiester, L., Hache, H., Muradyan, A., Schütte, M., Lim, J.-H., Heinig, M., et al. (2018) · Cell Systems · doi:10.1016/j.cels.2018.10.013 · Cited by 143
“Still, several concerns regarding reproducibility of results have been raised (Hafner et al., 2017; Niepel et al., 2017).”
BackgroundChemistry-First Approach for Nomination of Personalized Treatment in Lung Cancer
McMillan, E. A., Ryu, M.-J., Diep, C. H., Mendiratta, S., Clemenceau, J. R., Vaden, R. M., Kim, J.-H., Motoyaji, T., Covington, K. R., Peyton, M., et al. (2018) · Cell · doi:10.1016/j.cell.2018.03.028 · Cited by 141
“Furthermore, we examined potential correlations of chemical sensitivities (AUC and ED50 metrics) with cell doubling times (Data S6) as differences in proliferation rates can make confounding contributions to chemical sensitivity profiles within cell panels (Hafner et al., 2017).”
A Multi-center Study on the Reproducibility of Drug-Response Assays in Mammalian Cell Lines
Niepel, M., Hafner, M., Mills, C. E., Subramanian, K., Williams, E. H., Chung, M., Gaudio, B., Barrette, A. M., Stern, A. D., Hu, B., et al. (2019) · Cell Systems · doi:10.1016/j.cels.2019.06.005 · Cited by 133
Self-citationGRcalculator: an online tool for calculating and mining dose–response data
Clark, N. A., Hafner, M., Kouril, M., Williams, E. H., Muhlich, J. L., Pilarczyk, M., Niepel, M., Sorger, P. K., & Medvedovic, M. (2017) · BMC Cancer · doi:10.1186/s12885-017-3689-3 · Cited by 121
Self-citationAlQuraishi, M., & Sorger, P. K. (2021) · Nature Methods · doi:10.1038/s41592-021-01283-4 · Cited by 96
Self-citationRecommendations toward a human pathway-based approach to disease research
Marshall, L. J., Austin, C. P., Casey, W., Fitzpatrick, S. C., & Willett, C. (2018) · Drug Discovery Today · doi:10.1016/j.drudis.2018.05.038 · Cited by 28
Metadata
- Authors
- Hafner, M., Niepel, M., & Sorger, P. K.
- Venue
- Nature Biotechnology
- Published
- 2017-06
- Volume
- 35
- Issue
- 6
- Pages
- 500-502
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- OpenAlex
- W2621616899
- Cited by
- 85
- References
- 6
- FWCI
- 9.43
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.