CoChat
Sign Up
Cited by 92Computational Drug Discovery MethodsMachine Learning in Materials ScienceHistory and advancements in chemistry

Bioactivity descriptors for uncharacterized chemical compounds

Open access
Bertoni, Martino, Duran-Frigola, Miquel, Badia-i-Mompel, Pau, Pauls, Eduardo, Orozco-Ruiz, Modesto, Guitart-Pla, Oriol, Alcalde, Víctor, Diaz, Víctor M. +5 more·2021·Nature Communications
DOI 10.1038/s41467-021-24150-4PMC PMC8225676
Open this paper in CoChat — free accountOpen PDF

Abstract

Chemical descriptors encode the physicochemical and structural properties of small molecules, and they are at the core of chemoinformatics. The broad release of bioactivity data has prompted enriched representations of compounds, reaching beyond chemical structures and capturing their known biological properties. Unfortunately, bioactivity descriptors are not available for most small molecules, which limits their applicability to a few thousand well characterized compounds. Here we present a collection of deep neural networks able to infer bioactivity signatures for any compound of interest, even when little or no experimental information is available for them. Our signaturizers relate to bioactivities of 25 different types (including target profiles, cellular response and clinical outcomes) and can be used as drop-in replacements for chemical descriptors in day-to-day chemoinformatics tasks. Indeed, we illustrate how inferred bioactivity signatures are useful to navigate the chemical space in a biologically relevant manner, unveiling higher-order organization in natural product collections, and to enrich mostly uncharacterized chemical libraries for activity against the drug-orphan target Snail1. Moreover, we implement a battery of signature-activity relationship (SigAR) models and show a substantial improvement in performance, with respect to chemistry-based classifiers, across a series of biophysics and physiology activity prediction benchmarks.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper presents 25 deep neural network “signaturizers” that infer Chemical Checker bioactivity signatures for any compound, including poorly characterized ones, by transferring similarity information across 25 bioactivity spaces. Inferred signatures improve biological navigation of chemical space, enrich libraries for Snail1 activity, and outperform chemistry-based classifiers across many activity-prediction benchmarks.

Key findings

  • The 25 Siamese neural network signaturizers produce 128D signatures for the 25 Chemical Checker spaces, trained on 10^7 molecule triplets and applied to ~800,000 CC molecules; for D1 transcription, coverage increased from 11,638 experimental molecules to 69,532 with SNN predictions (498% increase).
  • Inferred signatures preserved cross-space correlations, and concatenated 3200D global signatures (GSigs) up-ranked compounds sharing Mode of Action or ATC codes and revealed biologically meaningful clusters in natural product and drug collections.
  • For Snail1, 222 compounds were selected from IRB/PWCK libraries; 22 of the top 25 strongest Snail1 down-regulators came from chemical/biological queries, selected compounds showed 5-fold enrichment over random, and 4 of the top 10 confirmed dose-dependent regulation.
  • Across 806 MoleculeNet tasks, SigAR models showed 8.5% median improvement over chemistry-based classifiers (IQR: 1.4–19.5%, Wilcoxon P = 5·10−60), with a median reduction of the gap to perfect performance of 17.6% (IQR: 24.4–31.5%).

Methods

  • For each CC space, a Siamese neural network was trained on anchor/positive/negative triplets using stacked available CC signatures as input, a signature-dropout scheme, and a 128D output; 10^7 triplets were used with 80:20 train–test splits and evaluated by triplet classification accuracy and Matthews correlation coefficient.
  • Large-scale inference generated 25 × 128D signatures for ~800,000 CC molecules, with an applicability score (α) based on distance to training signatures, robustness to test-time dropout, and a priori expectancy; light-weight chemistry-to-signature signaturizers were also trained using 2048-bit Morgan fingerprints.
  • Snail1 enrichment used chemical and biological queries on IRB/PWCK libraries followed by a Firefly:Renilla luciferase assay in MDA-MB-231 cells; SigAR models used random forest classifiers with conformal prediction on MoleculeNet tasks, compared against Morgan fingerprints and other chemical descriptors.

Limitations

  • Stated: CC signatures are difficult to interpret in detail because underlying data points cannot be deconvoluted from the 128D signature; null signatures for innocuous molecules may exist, and signature accuracy varies by molecule, so an applicability score accompanies predictions.
  • Stated: growth of the CC resource is restricted by the number and quality of publicly accessible datasets, and the current version is constrained to 25 categories, though it is prepared to accommodate new data types.
  • Evident from the text: cell-based and clinical signaturizers were the most challenging, with cell-based accuracy moderate on average (~0.7) and clinical performance variable; benchmark comparisons may also be affected by shared data between MoleculeNet and CC, although the authors excluded certain CC signature classes for some tasks.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41467-021-24150-4

Cites

Statistics

Nothing to recompute.

Claims the paper makes

0 of 20 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Compounds identified by the chemical query are more similar to the reference set of 45 known DUB inhibitors than those retrieved by the biological queries.

    “As expected, compounds identified by the chemical query are more similar to our reference set of 45 known DUB inhibitors than those retrieved by the biological queries (Supplementary Fig. 13b).”
    Quote in the paperNot a substantial claim
  • Findingown data

    Bioactivity maps were well covered by test-set molecules.

    “Overall, bioactivity maps were well covered by test-set molecules, indicating that our SNNs are unbiased and able to generate predictions that are spread throughout the complete bioactivity landscape (Fig. 1g and Supplementary Fig. 5).”
    Quote in the paperSubstantial claim
  • Findingown data

    The original pattern of correlations between the 25 data types was preserved among inferred signatures, especially for the high-α ones.

    “The original pattern of correlations was preserved among inferred signatures, especially for the high-α ones (Fig. 2c and Supplementary Fig. 9)”
    Quote in the paperSubstantial claim
  • Findingown data

    Most of the clusters have a mixed composition, containing subgroups of chemically related compounds while also including distinct molecules.

    “Most of the clusters have a mixed composition, containing subgroups of chemically related compounds while also including distinct molecules, as is the case for the HSP90AA1-associated cluster, of which compounds 4 and 5 are good representatives (Fig. 2h).”
    Quote in the paperNot a substantial claim
  • Findingown data

    In a similarity search exercise, α scores ≥0.5 retrieved a significant number of true hits (odds-ratios > 8, P-values < 1.7·10−21).

    “In a similarity search exercise, we found that α scores ≥0.5 retrieved a significant number of true hits (odds-ratios > 8, P-values < 1.7·10−21 (Supplementary Fig. 7)).”
    Quote in the paperSubstantial claim
  • May combine two claims. 47
  • Classified as probably not a substantial claim. 25

  • May combine two claims.

    Findingown data

    Some of the clusters group molecules with similar chemistries (e.g., ESR1,2 ligands), while others correspond to sets of diverse compounds (e.g., MAPK8,9,10 inhibitors).

    “Of note, some of the clusters group molecules with similar chemistries (e.g., ESR1,2 ligands), while others correspond to sets of diverse compounds (e.g., MAPK8,9,10 inhibitors).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    At the targets levels (B), the performance of the signaturizers was high for large-scale binding data B4, while accuracy was variable at deeper annotation levels where the number of compounds available for training was smaller.

    “At the targets levels (B), the performance of the signaturizers was high for large-scale binding data B4, while accuracy was variable at deeper annotation levels where the number of compounds available for training was smaller (e.g., MoA (B1) or for drug-metabolizing enzymes (B2)) (Fig. 1d).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Selected compounds had comparable molecular weights and drug-like properties.

    “Selected compounds had comparable molecular weights and drug-like properties (Supplementary Fig. 13a).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    We retrieved 95 transcriptional signatures from the L1000 Connectivity Map and 18 from the Gene Expression Omnibus (GEO).

    “In total, we retrieved 95 transcriptional signatures from the L1000 Connectivity Map and 18 from the Gene Expression Omnibus (GEO)25”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [25] Barrett, T. et al. NCBI GEO: archive for functional genomics data sets–update. Nucleic Acids Res. 41, D991–D995 (2013). source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    The PWCK library is composed of well-annotated molecules over a wide range of activities, with >99% of its molecules present in the CC.

    “The PWCK library is considerably smaller (>1,000 compounds), and it is composed of well-annotated molecules over a wide range of activities (>99% of the molecules are present in the CC).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The inferred A-level signatures faithfully reproduce the original CC signatures and recapitulate the similarity information captured by ECFP4.

    “we see that the inferred signatures cannot only faithfully reproduce the original CC signatures but also recapitulate the similarity information captured by the much longer extended connectivity fingerprints (ECFP4; Supplementary Fig. 4).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Drug-related libraries (e.g., IUPHAR and IDG) had similar occupancy vectors to the reference CC library, meaning they were evenly distributed in the bioactivity space.

    “As can be seen, drug-related libraries (e.g., IUPHAR and IDG) had similar occupancy vectors to the reference CC library, meaning they were evenly distributed in the bioactivity space, which is expected given the over-representation of medicinal chemistry in our resource.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingnone given

    The IRB library contains >17,000 compounds, only 3% of which have reported bioactivities and are thus included in the CC.

    “The IRB library contains >17,000 compounds, only 3% of which have reported bioactivities and are thus included in the CC.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    We curated 45 DUB inhibitors, 6 of which were inhibitors of candidate DUBs in the siRNA-DUB/Snail1 assay.

    “We curated 45 DUB inhibitors, 6 of which were inhibitors of candidate DUBs in the siRNA-DUB/Snail1 assay.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Libraries containing building blocks from different providers (ChemDiv, Sigma-Aldrich, and ChemBridge) were grouped together, although with an uneven representativity of the CC bioactivity space.

    “Libraries containing BBs from different providers (ChemDiv, Sigma-Aldrich, and ChemBridge) were grouped together, although with an uneven representativity of the CC bioactivity space.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    Overall, we selected 222 compounds from the three queries; 131 from the IRB library and 91 from the PWCK collection.

    “Thus, overall, we selected 222 compounds from the three queries (Supplementary Data 1); 131 from the IRB library and 91 to the PWCK collection.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Similarity measures performed in the GSig space up-rank pairs of compounds with the same MoA or ATC code and have an overall correlation with the rest of experimental data available from the CC, capturing not only chemical similarities between molecules but also common target profiles, clinical characteristics and, to a lesser degree, cell-based assay read-outs.

    “Similarity measures performed in the GSig space up-rank pairs of compounds with the same MoA or ATC code (Fig. 2e) and have an overall correlation with the rest of experimental data available from the CC, capturing not only chemical similarities between molecules but also common target profiles, clinical characteristics and, to a lesser degree, cell-based assay read-outs (Fig. 2f).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Short distances between molecule signatures reflect similar biological behavior.

  • May combine two claims.

    Findingown data

    Chemistry (A) signatures are widely available (~106 compounds), whereas cell-based assays (D) cover about 30,000 molecules, and clinical (E) signatures are known for only a few thousand drugs.

    “As a result, chemistry (A) signatures are widely available (~106 compounds), whereas cell-based assays (D) cover about 30,000 molecules, and clinical (E) signatures are known for only a few thousand drugs (Fig. 1b).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PI3K, CDK, and VEGFR inhibitors have their own well-defined clusters within the oncology region of the map, and histamine receptor antagonists and acetylcholine receptor agonists are placed together in an area assigned to neurology/psychiatry.

    “This is exemplified by PI3K, CDK, and VEGFR inhibitors, which have their own well-defined clusters within the oncology region of the map, and by histamine receptor antagonists and acetylcholine receptor agonists, which are placed together in an area assigned to neurology/psychiatry (Fig. 3d and Supplementary Fig. 11).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Broad therapeutic areas such as infectious diseases, neurology/psychiatry, cardiology, and oncology can be circumscribed within certain regions of the GSig landscape, and the same applies to finer-grained disease categories (indications) and mechanisms of action.

    “For example, when we focus on drug molecules (REPO Hub), broad therapeutic areas such as infectious diseases, neurology/psychiatry, cardiology, and oncology can be circumscribed within certain regions of the GSig landscape (Fig. 3d), and the same applies to finer-grained disease categories (indications) and mechanisms of action (Supplementary Fig. 11).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Low- and high-α areas of the signature landscape can be easily delimited, indicating the presence of reliable regions in the prediction space.

    “Moreover, low- and high-α areas of the signature landscape can be easily delimited, indicating the presence of reliable regions in the prediction space (Fig. 2b).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The chemical and biological queries retrieved 169 and 131 compounds, respectively, with 78 molecules being picked by both.

    “After inferring CC bioactivity signatures for all the ~20,000 compounds in our libraries, the chemical and biological queries detailed above retrieved 169 and 131 compounds, respectively, with 78 molecules being picked by both.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The performance of SNNs varied depending on the CC space and molecule of interest, with signatures being well predicted in all spaces.

    “The performance of SNNs varied depending on the CC space and molecule of interest, with signatures being well predicted in all spaces.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The map reveals a region that is specific to drug molecules, possibly belonging to a set of bioactivities that is outside the reach of natural metabolites.

    “The map reveals a region that is specific to drug molecules, possibly belonging to a set of bioactivities that is outside the reach of natural metabolites.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On average, the accuracy of cell-based signaturizers was moderate (~0.7) and true-vs-predicted correlation of clinical signatures such as therapeutic classes (ATC; E1) was variable across molecules.

    “On average, the accuracy of cell-based signaturizers was moderate (~0.7) and true-vs-predicted correlation of clinical signatures such as therapeutic classes (Anatomical Therapeutic Chemical, ATC; E1) was variable across molecules.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The NR-AhR model mostly leverages the chemical levels (A), whereas SR-ATAD5 benefits from cell sensitivity data (D2), and NR-ER-LBD exploits the functional (e.g., biological process (C3)) information contained within the network levels of the CC.

    “The NR-AhR model, for instance, mostly leverages the chemical levels (A), whereas SR-ATAD5 benefits from cell sensitivity data (D2), and NR-ER-LBD exploits the functional (e.g., biological process (C3)) information contained within the network levels of the CC.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The three compounds retrieved only by biological queries are chemically unrelated to any of the known DUB inhibitors, with Tanimoto similarities ranging from 0.13 to 0.32.

    “It is important to note that these three compounds are chemically unrelated to any of the known DUB inhibitors, with Tanimoto similarities ranging from 0.13 to 0.32 (Supplementary Data 1).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    A substantial number of hits (6 in the top 25) were candidate molecules selected by both biological and chemical queries, and an additional 3 compounds were retrieved only by biological queries.

    “Importantly, a substantial number of hits (6 in the top 25) were candidate molecules selected by both biological and chemical queries, and an additional 3 compounds were retrieved only by biological queries (Supplementary Fig. 13c).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Compounds 8 and 9 had the same chemotype, which was identified in 4 of the top 25 hits.

    “Of note, compounds 8 and 9 had the same chemotype, which was identified in 4 of the top 25 hits.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Some molecules had a high prediction score with the GSig-based model but were predicted to be inactive by the MFp-based counterpart, and vice versa.

    “Of note, some molecules had a high prediction score with the GSig-based model but were nonetheless predicted to be inactive by the MFp-based counterpart, and vice versa (Fig. 5c)”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    CC chemistry levels were not among the best explanatory signature types for the SR-HSE classifier; HTS bioassays (B5) and cell morphology data (D4) appeared to be more informative.

    “Indeed, CC chemistry levels were not among the best explanatory signature types for the SR-HSE classifier. Instead, HTS bioassays (B5) and cell morphology data (D4) appeared to be more informative (Fig. 5d)”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Scrambled signatures are different from real GSigs and contain no bioactivity signal.

    “scrambled signatures are indeed different from real GSigs and contain no bioactivity signal.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When using random molecules as bait, true neighbors cannot be identified and there is no relationship with applicability scores.

    “when we use random molecules as bait, we cannot identify true neighbors and there is no relationship with the applicability scores.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Signatures that are close to training-set signatures are, in principle, closer to the applicability domain.

  • May combine two claims.

    Findingreasoning

    17 of the positive hits were not known to be bioactive, and therefore their CC signatures have been fully inferred by our signaturizers.

    “It is also worth noting that 17 of the positive hits were not known to be bioactive, and therefore their CC signatures have been fully inferred by our signaturizers.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    GSigs are generally favorable to MFps across the 12 toxicity pathways defined in the Tox21 benchmark data set, with particularly large differences for the SR-p53, NR-Aromatase, NR-AR, NR-PPAR-gamma, and SR-HSE tasks, and essentially the same performance for the NR-AhR and NR-ER tasks.

    “Figure 5f demonstrates that GSigs are generally favorable to MFps across the 12 toxicity pathways defined in the Tox21 benchmark data set, with particularly large differences for the SR-p53, NR-Aromatase, NR-AR, NR-PPAR-gamma, and SR-HSE tasks, and essentially the same performance for the NR-AhR and NR-ER tasks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The CC is limited by the availability of experimental data and the coverage of its source databases.

    “In essence, the CC is an alternative representation of the small-molecule knowledge deposited in the public domain and, as such, it is also limited by the availability of experimental data and the coverage of its source databases (e.g., ChEMBL9 or DrugBank10).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [9] Mendez, D. et al. ChEMBL: towards direct deposition of bioassay data. Nucleic Acids Res. 47, D930–D940 (2019). source

      Not checked in the free run.

    • cites [10] Wishart, D. S. et al. DrugBank 5.0: a major update to the DrugBank database for 2018. Nucleic Acids Res. 46, D1074–D1082 (2018). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Public databases contain experimentally determined bioactivity data for about a million molecules, which represent only a small percentage of commercially available compounds and a negligible fraction of the synthetically accessible chemical space.

    “Currently, public databases contain experimentally determined bioactivity data for about a million molecules, which represent only a small percentage of commercially available compounds6 and a negligible fraction of the synthetically accessible chemical space7.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [6] Sterling, T. & Irwin, J. J. ZINC 15–ligand discovery for everyone. J. Chem. Inf. Model 55, 2324–2337 (2015). source

      Not checked in the free run.

    • cites [7] Reymond, J.-L. The chemical space project. Acc. Chem. Res. 48, 722–730 (2015). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The CC is most useful when a substantial amount of bioactivity information is available for the molecules and remains of limited value for poorly characterized compounds.

    “Thus, the CC is most useful when a substantial amount of bioactivity information is available for the molecules and remains of limited value for poorly characterized compounds11.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [11] Duran-Frigola, M., Mateo, L. & Aloy, P. Drug repositioining beyond the low-hanging fruits. Curr. Opin. Syst. Biol. 3, 2452–3100 (2017).

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Bioactivity signatures cannot be derived for most compounds, and CDD methods are limited to using chemical information alone as a primary input, thereby hindering their performance and not fully exploiting the bioactivity knowledge produced over the years by the scientific community.

    “In practical terms, this means bioactivity signatures cannot be derived for most compounds, and CDD methods are limited to using chemical information alone as a primary input, thereby hindering their performance and not fully exploiting the bioactivity knowledge produced over the years by the scientific community.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Inferred bioactivity signatures are useful to navigate the chemical space in a biologically relevant manner, unveiling higher-order organization in natural product collections.

    “we illustrate how inferred bioactivity signatures are useful to navigate the chemical space in a biologically relevant manner, unveiling higher-order organization in natural product collections”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Chemical Checker (CC) is the largest collection of small-molecule bioactivity signatures available to date.

    “Recently, we integrated the major chemogenomics and drug databases in a single resource named the Chemical Checker (CC), which is the largest collection of small-molecule bioactivity signatures available to date8.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8] Duran-Frigola, M. et al. Extending the small-molecule similarity principle to all levels of biology with the Chemical Checker. Nat. Biotechnol. 38, 1087–1096 (2020). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    The different bioactivity spaces are not completely independent.

    “based on the observation that the different bioactivity spaces are not completely independent”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Snail1 is a zinc-finger transcription factor with an essential role in the epithelial-to-mesenchymal transition (EMT).

    “Snail1, a zinc-finger transcription factor with an essential role in the epithelial-to-mesenchymal transition (EMT)18.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [18] Dongre, A. & Weinberg, R. A. New insights into the mechanisms of epithelial-mesenchymal transition and implications for cancer. Nat. Rev. Mol. Cell Biol. 20, 69–84 (2019). source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    A signaturizer takes as input the subset of CC signatures available for a molecule and produces a 128D signature that captures the similarity profile of the molecule in the Si CC space, where experimental information may not be available for the compound.

    “A signaturizer takes as input the subset of CC signatures available for a molecule and produces a 128D signature that, in principle, captures the similarity profile of the molecule in the Si CC space, where experimental information may not be available for the compound.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Short distances between molecule signatures reflect similar biological behavior.

  • May combine two claims.

    Interpretationown data

    The chemistry-to-clinics scope of GSigs provides a multi-level view of the chemical space, clustering compounds first on the basis of their targets and, in turn, keeping targets close in space if they belong to the same disease area.

    “Thus, the chemistry-to-clinics scope of GSigs provides a multi-level view of the chemical space, clustering compounds first on the basis of their targets and, in turn, keeping targets close in space if they belong to the same disease area.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning

    The networks level (C) is directly informed by the underlying targets (B) level.

    “Performance at the networks level (C) was high, as this level is directly informed by the underlying targets (B) level.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Knock-down of certain deubiquitinases significantly decreased Snail1 levels, suggesting that DUBs promote Snail1 stabilization and are required for its effects on EMT and cancer progression.

    “In a previous siRNA screening, we found that the knock-down of certain deubiquitinases (DUBs) significantly decreased Snail1 levels, suggesting that DUBs promote Snail1 stabilization and are required for its effects on EMT and cancer progression20.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [20] Lambies, G. et al. TGFbeta-activated USP27X deubiquitinase regulates cell migration and chemoresistance via stabilization of Snail1. Cancer Res. 79, 33–46 (2019). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Our SNNs are unbiased and able to generate predictions that are spread throughout the complete bioactivity landscape.

    “Overall, bioactivity maps were well covered by test-set molecules, indicating that our SNNs are unbiased and able to generate predictions that are spread throughout the complete bioactivity landscape (Fig. 1g and Supplementary Fig. 5).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    CC signatures offer a novel bioactivity-driven means to organize chemical space, with the potential to unveil higher levels of organization that may not be apparent in the light of chemical information alone.

    “Taken together, CC signatures offer a novel bioactivity-driven means to organize chemical space, with the potential to unveil higher levels of organization that may not be apparent in the light of chemical information alone.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The IRB and PWCK libraries represent two typical scenarios: the recycling of a targeted library, and the use of a small diversity-oriented compound collection, respectively.

    “Thus, the IRB and PWCK libraries represent two typical scenarios: the recycling of a targeted library, and the use of a small diversity-oriented compound collection, respectively.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    We assigned an applicability score (α) to predictions based on the proximity of a predicted signature to true signatures available in the training set, the robustness of the SNN output to a test-time data dropout, and the accuracy expected a priori based on the experimental CC data sets available for the molecule.

    “To explore the reliability of the inferred signatures, we assigned an applicability score (α) to predictions based on the following: (a) the proximity of a predicted signature to true (experimental) signatures available in the training set; (b) the robustness of the SNN output to a test-time data dropout12; and (c) the accuracy expected a priori based on the experimental CC data sets available for”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [12] Cortes-Ciriano, I. & Bender, A. Reliable prediction errors for deep neural networks using test-time dropout. J. Chem. Inf. Model 59, 3330–3339 (2019). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    A-level signatures are of little added value, since chemical information is always available for compounds.

    “The A-level signatures are of little added value, since chemical information is always available for compounds”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisereasoning

    Biological pathway signatures (C3) are directly derived from binding signatures (B4), thus implying that, in a real B4 prediction case, C3 will never serve as a covariate.

    “For example, in the CC, biological pathway signatures (C3) are directly derived from binding signatures (B4), thus implying that, in a real B4 prediction case, C3 will never serve as a covariate.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    The ever-growing nature of chemical matter (in contrast to the finite number of genes) demands computational methods to provide a first estimate of the biological properties of compounds.

    “The ever-growing nature of chemical matter (in contrast to the finite number of genes) demands computational methods to provide a first estimate of the biological properties of compounds41.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [41] Vamathevan, J. et al. Applications of machine learning in drug discovery and development. Nat. Rev. Drug Discov. 18, 463–477 (2019). source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The authors followed MoleculeNet recommendations strictly for splitting and performance measures to ensure a fair assessment.

    “we followed MoleculeNet recommendations strictly, both in terms of splitting (e.g., scaffold-based) and in terms of performance measure (AUROC/AUPR) to ensure a fair assessment.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Classifiers were calibrated using a Mondrian cross-conformal prediction scheme over 10 stratified splits.

    “Classifiers were calibrated using a Mondrian cross-conformal prediction scheme over 10 stratified splits.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Contrary to most chemical descriptors, CC signatures evolve with time as bioactivity measurements accumulate in the databases.

    “Contrary to most chemical descriptors, CC signatures evolve with time as bioactivity measurements accumulate in the databases.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    CC signatures are difficult to interpret in detail; the underlying data points cannot be deconvoluted from the 128D signature.

    “A limitation of CC signatures is that they are difficult to interpret in detail. That is, the underlying data points (binding to receptor x, occurrence of drug side effect y, etc.) cannot be deconvoluted from the 128D signature.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    This caveat is common to other machine-learning applications where embedded representations of entities are favored over sparser, more explicit ones.

    “This caveat is common to other machine-learning applications (e.g., natural language processing) where embedded representations of entities are favored over sparser, more explicit ones37.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37] Duran-Frigola, M., Fernandez-Torras, A., Bertoni, M. & Aloy, P. Formatting biological big data for modern machine learning in drug discovery. WIREs Comput. Mol. Sci. 9, 1759–0876 (2019). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The evaluation was done with five stratified 80:20 train-test splits.

    “The evaluation was done with five stratified 80:20 train–test splits.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    There is likely existence of null signatures corresponding to innocuous molecules with no actual bioactivity in a given CC data type.

    “Another caveat of our approach is the likely existence of null signatures corresponding to innocuous molecules with no actual bioactivity in a given CC data type38.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [38] Subramanian, A. et al. A next generation connectivity map: L1000 Platform and the first 1,000,000 profiles. Cell 171, 1437–1452.e17 (2017). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    CC signatures can be used to navigate the chemical space in a biological-relevant manner, revealing unexpected high-order structure in poorly annotated natural product collections.

    “In this study, we showed how CC signatures can be used to navigate the chemical space in a biological-relevant manner, revealing somehow unexpected high-order structure in poorly annotated natural product collections.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    CTS signaturizers were validated with an 80:20 train-test split.

    “CTS signaturizers were trained for 30 epochs and validated with an 80:20 train–test split”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    We repeated the experiment with a model-agnostic approach based on an AutoML methodology to confirm that the observed trends are not dependent on the random forest classifier.

    “Nevertheless, to confirm that the observed trends are not dependent on the random forest classifier, we repeated the experiment with a model-agnostic approach based on an AutoML methodology.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    We followed MoleculeNet recommendations strictly, both in terms of splitting (e.g., scaffold-based) and in terms of performance measure (AUROC/AUPR) to ensure a fair assessment.

    “To evaluate the accuracy of the classifiers, we followed MoleculeNet recommendations strictly, both in terms of splitting (e.g., scaffold-based) and in terms of performance measure (AUROC/AUPR) to ensure a fair assessment.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The accuracy of the signatures may vary depending on the molecule.

    “Likewise, the accuracy of the signatures may vary depending on the molecule.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    Although the current version of the CC is constrained to 25 categories, the resource is prepared to accommodate new data types, offering the opportunity to customize and extend the current repertoire of signaturizers.

    “Although the current version of the CC is constrained to 25 categories, our resource is prepared to accommodate new data types, offering the opportunity to customize and extend the current repertoire of signaturizers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The luciferase inhibition predictor was evaluated with 5-fold stratified cross-validation.

    “We performed a 5-fold stratified cross-validation”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Near-duplicates were removed with FAISS before splitting into train and test sets.

    “Splitting was done after removing near-duplicates with FAISS.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The SigAR task predictions were repeated with a model-agnostic AutoML TPOT approach to provide a fair ML-independent comparison.

    “we repeated the SigAR task predictions with a model-agnostic approach based on the AutoML TPOT methodology”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The PCBA MoleculeNet subtasks could not be addressed due to prohibitive computational costs for TPOT.

    “we could not address the PCBA MoleculeNet subtasks (involving 400k molecules) due to prohibitive computational costs for TPOT.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Four replicas of the dual-luciferase assay were conducted on two days.

    “Four replicas conducted on two days were performed.”
    Quote in the paperNot a substantial claim
67 of 67

  • The paper relies on this without support.

    Short distances between molecule signatures reflect similar biological behavior.

    “Bioactivity signatures must be amenable to similarity calculations, ideally by conventional metrics such as cosine or Euclidean distances, so that short distances between molecule signatures reflect a similar biological behavior.”

  • The paper relies on this without support.

    Data points can be shared between MoleculeNet and the CC, which would trivialize predictions.

    “Although none of these benchmark data sets are explicitly included in the CC resource, data points can be shared between MoleculeNet and the CC, which would trivialize predictions.”

  • The paper relies on this without support.

    Signatures that are close to training-set signatures are, in principle, closer to the applicability domain.

    “Signatures that are close to training-set signatures are, in principle, closer to the applicability domain.”

  • The paper relies on this without support.

    Small standard deviations over dropped-out predictions indicate a robust output.

    “Small standard deviations over these predictions indicate a robust output.”

  • The paper relies on this without support.

    Some CC signature types are highly predictive for others; thus, having these informative signatures at hand will in principle favor reliable predictions.

    “Some CC signature types are highly predictive for others; thus, having these informative signatures at hand will in principle favor reliable predictions.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A broad set of traditional and modern small-molecule descriptors can capture physicochemical properties, 2D/3D structural information, and a priori knowledge of compound bioactivities.

    “Likewise, a broad set of traditional and modern small-molecule descriptors are able to capture physicochemical properties, 2D/3D structural information, and even a priori knowledge of compound bioactivities,192,193”

    From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [193]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Conflicting hidden-layer sizes for m = 7

    Major

    For m = 7, the input has 7 × 128 = 896 dimensions, exceeding the stated 512-dimension threshold. The rule therefore specifies hidden layers of 512 and 256, but the example gives 448 and 224.

  • Biological processes assigned to two different CC spaces

    Minor

    The Snail1 analysis assigns biological processes to C4, whereas the benchmark discussion labels biological process information as C3.

  • Reported median falls outside its interquartile range

    Minor

    A median of 17.6% cannot have an interquartile range of 24.4-31.5%; the median must lie between the first and third quartiles.

3 of 3

The citation lists haven’t been fetched yet

References 50 · Cited by 92

Research works printed but not linked: 1

  1. Duran-Frigola, M., Mateo, L. & Aloy, P. Drug repositioining beyond the low-hanging fruits. Curr. Opin. Syst. Biol. 3, 2452–3100 (2017).

Authors
Bertoni, M., Duran-Frigola, M., Badia-i-Mompel, P., Pauls, E., Orozco-Ruiz, M., Guitart-Pla, O., Alcalde, V., Diaz, V. M., Berenguer-Llergo, A., Brun-Heath, I., Villegas, N., de Herreros, A. G., & Aloy, P.
Affiliations
  • Institute for Research in Biomedicine (ES)
  • Universitat Pompeu Fabra (ES)
  • Institució Catalana de Recerca i Estudis Avançats (ES)
Corresponding author
Aloy, Patrick
ORCID
Venue
Nature Communications
Publisher
Springer Science and Business Media LLC
Published
2021-06-24
Volume
12
Issue
1
Pages
3932-3932
ISSN
2041-1723 (ISSN-L)
DOI
10.1038/s41467-021-24150-4
PMC
PMC8225676
OpenAlex
W3173289526
Semantic Scholar
50a139e79b4c31782d5b82630ecdb9860b3cd16b
Cited by
92
References
50
Citations by year
2026: 11 · 2025: 20 · 2024: 13 · 2023: 23 · 2022: 19 · 2021: 6
FWCI
8.04
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Repositori digital de la UPF (Universitat Pompeu Fabra) (publishedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Institute for Research in Biomedicine
  • European Commission (101003633)
  • Generalitat de Catalunya (001-P-001682, 001-P-001647, PID2019)
  • Ministerio de Economía y Competitividad (PID2019, BIO2016‐77038‐R)
  • Agencia Estatal de Investigación (BIO2016-77038-R, PID2019-104698RB-I00, PID2019)
  • FP7 Ideas: European Research Council (614944)
Topics
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
  • Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
  • Physical Sciences › Chemistry › Physical and Theoretical Chemistry › History and advancements in chemistry
Keywords
molecular descriptors, deep neural network, bioactivity prediction, cheminformatics, SNAI1, cellular response, clinical outcomes
MeSH terms
Snail Family Transcription Factors, Drug Evaluation, Preclinical, Humans, Structure-Activity Relationship, Cell Line, Tumor, Small Molecule Libraries, Databases, Pharmaceutical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review