CoChat
Sign Up
Cited by 109Computational Drug Discovery MethodsNeural Networks and ApplicationsAdvanced Memory and Neural Computing

The DOI matches its Crossref record; 129 of the 129 references it cites were found in a registry or an index.

Deep Artificial Neural Networks and Neuromorphic Chips for Big Data Analysis: Pharmaceutical and Bioinformatics Applications

Open access
Pastur-Romay, Lucas, Cedrón, Francisco, Pazos, Alejandro, Porto-Pazos, Ana·2016·International Journal of Molecular Sciences
DOI 10.3390/ijms17081313PMC PMC5000710
Open this paper in CoChat — free accountOpen PDF

Abstract

Over the past decade, Deep Artificial Neural Networks (DNNs) have become the state-of-the-art algorithms in Machine Learning (ML), speech recognition, computer vision, natural language processing and many other tasks. This was made possible by the advancement in Big Data, Deep Learning (DL) and drastically increased chip processing abilities, especially general-purpose graphical processing units (GPGPUs). All this has created a growing interest in making the most of the potential offered by DNNs in almost every field. An overview of the main architectures of DNNs, and their usefulness in Pharmacology and Bioinformatics are presented in this work. The featured applications are: drug design, virtual screening (VS), Quantitative Structure–Activity Relationship (QSAR) research, protein structure prediction and genomics (and other omics) data mining. The future need of neuromorphic hardware for DNNs is also discussed, and the two most advanced chips are reviewed: IBM TrueNorth and SpiNNaker. In addition, this review points out the importance of considering not only neurons, as DNNs and neuromorphic chips should also include glial cells, given the proven importance of astrocytes, a type of glial cell which contributes to information processing in the brain. The Deep Artificial Neuron–Astrocyte Networks (DANAN) could overcome the difficulties in architecture design, learning process and scalability of the current ML methods.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/ijms17081313

Cites

129 of 129 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Big Data Workflow.
Figure 1 Big Data Workflow.CC BY · PubMed Central
Figure 2 Deep neural network architecture from Yanjun Qi et al. [45]. The input to the first layer is the protein sequence represented by the single-letter amino acid code, for example the letter “A” (in green) represents “Alanine”. This method uses a sliding window input {S1, S2… Sk}, in this case k = 7. The first layer consists a PSI-Blast feature module and an amino acid embedding module, the green boxes represent the feature vector derived from the Alanine in both modules. In the second layer, the feature vectors are concatenated to facilitate identification of local sequence structure. Finally the derived vector is fed into the Deep Artificial Neural Network.
Figure 2 Deep neural network architecture from Yanjun Qi et al. [45]. The input to the first layer is the protein sequence represented by the single-letter amino acid code, for example the letter “A” (in green) represents “Alanine”. This method uses a sliding window input {S1, S2… Sk}, in this case k = 7. The first layer consists a PSI-Blast feature module and an amino acid embedding module, the green boxes represent the feature vector derived from the Alanine in both modules. In the second layer, the feature vectors are concatenated to facilitate identification of local sequence structure. Finally the derived vector is fed into the Deep Artificial Neural Network.CC BY · PubMed Central
Figure 3 Convolutional layers that extract features of the input to create a feature map. The artificial neurons are represented by the circles, and the weights by the narrows. Weights of the same color are shared, constrained to be identical [56].
Figure 3 Convolutional layers that extract features of the input to create a feature map. The artificial neurons are represented by the circles, and the weights by the narrows. Weights of the same color are shared, constrained to be identical [56].CC BY · PubMed Central
Figure 4 Architecture of a Deep Convolutional Neural Network (DCNN), alternating the convolutional layer and the max-pooling layer (or sub-sampling layer), and finally the fully-connected layer [56].
Figure 4 Architecture of a Deep Convolutional Neural Network (DCNN), alternating the convolutional layer and the max-pooling layer (or sub-sampling layer), and finally the fully-connected layer [56].CC BY · PubMed Central
Figure 5 This diagram represents a simplification of the structure of the epoxidation model, which was made up of one input layer, two hidden layers, and two output layers. The actual model had several additional nodes in the input and hidden layers. In the input layer, M represents the molecule input node, B is the bond input node, and two atom input nodes (for each atom associated with the bond). The bond epoxidation score (BES) quantifies the probability that the bond is a site of epoxidation based in the input from the nodes of the first hidden layer (H1 and H2). The molecule epoxidation score (MES) reflects the probability that the molecule will be epoxidized. This score is calculated with the information from the all molecule-level descriptors and the BES. The bond network and the molecule network are represented in orange and purple respectively [57].
Figure 5 This diagram represents a simplification of the structure of the epoxidation model, which was made up of one input layer, two hidden layers, and two output layers. The actual model had several additional nodes in the input and hidden layers. In the input layer, M represents the molecule input node, B is the bond input node, and two atom input nodes (for each atom associated with the bond). The bond epoxidation score (BES) quantifies the probability that the bond is a site of epoxidation based in the input from the nodes of the first hidden layer (H1 and H2). The molecule epoxidation score (MES) reflects the probability that the molecule will be epoxidized. This score is calculated with the information from the all molecule-level descriptors and the BES. The bond network and the molecule network are represented in orange and purple respectively [57].CC BY · PubMed Central
Figure 6 Details of inner workings of DeepBind developed by Alipanahi et al. and its training procedure. In “a”, five independent sequences of DNA are being processed in parallel, each composed by a string of letters (C, G, A and T) which represent the nucleotides. The scores are represented in white and red tones, and the outputs are compared to the targets to improve the model using backpropagation; In “b”, The Calibration, training, and tasting procedure is represented in more detail [59].
Figure 6 Details of inner workings of DeepBind developed by Alipanahi et al. and its training procedure. In “a”, five independent sequences of DNA are being processed in parallel, each composed by a string of letters (C, G, A and T) which represent the nucleotides. The scores are represented in white and red tones, and the outputs are compared to the targets to improve the model using backpropagation; In “b”, The Calibration, training, and tasting procedure is represented in more detail [59].CC BY · PubMed Central
Figure 7 Different Recurrent Neural Networks architectures, the white circles represent the input layers, the black circles the hidden layers, and the grey circles the output layers [65].
Figure 7 Different Recurrent Neural Networks architectures, the white circles represent the input layers, the black circles the hidden layers, and the grey circles the output layers [65].CC BY · PubMed Central
Figure 8 Schematic diagram of Youjun Xu et al. network encoding glycine, first using primary canonical SMILES strucuture. Then, each of the atoms in the SMILES structure is sequentially defined as a root node. Finally, information for all other atoms is transferred along the shortest possible paths, in which case is obtained following the narrows [67].
Figure 8 Schematic diagram of Youjun Xu et al. network encoding glycine, first using primary canonical SMILES strucuture. Then, each of the atoms in the SMILES structure is sequentially defined as a root node. Finally, information for all other atoms is transferred along the shortest possible paths, in which case is obtained following the narrows [67].CC BY · PubMed Central
Figure 9 Mapping a Deep Artificial Neural Network (DANN) (a) to a neuromorphic chip like the TrueNorth (b). The input neurons are represented with the red and white shapes (x and x’), and the output neurons with the grey shapes (z and z’). The weights (w) to the neuron z are approximated using a Pseudo Random Number Generator (PRNG), resulting in the weights (w’) to the neuron z’ in the neuromorphic chip [74].
Figure 9 Mapping a Deep Artificial Neural Network (DANN) (a) to a neuromorphic chip like the TrueNorth (b). The input neurons are represented with the red and white shapes (x and x’), and the output neurons with the grey shapes (z and z’). The weights (w) to the neuron z are approximated using a Pseudo Random Number Generator (PRNG), resulting in the weights (w’) to the neuron z’ in the neuromorphic chip [74].CC BY · PubMed Central
Figure 10 (A) The neurosynaptic core is loosely inspired by the idea of a canonical cortical microcircuit; (B) A network of neurosynaptic cores is inspired by the cortex’s two-dimensional sheet, the brain regions are represented in different colors; (C) The multichip network is inspired by the long-range connections between cortical regions shown from the macaque brain; (D–F) Structural scheme of the core, chip and multi-chip level. The white shapes represent axons (inputs) and the grey shapes the neurons (outputs); (G–I) Functional view at different level; (J–L) Image of the physical layout [77].
Figure 10 (A) The neurosynaptic core is loosely inspired by the idea of a canonical cortical microcircuit; (B) A network of neurosynaptic cores is inspired by the cortex’s two-dimensional sheet, the brain regions are represented in different colors; (C) The multichip network is inspired by the long-range connections between cortical regions shown from the macaque brain; (D–F) Structural scheme of the core, chip and multi-chip level. The white shapes represent axons (inputs) and the grey shapes the neurons (outputs); (G–I) Functional view at different level; (J–L) Image of the physical layout [77].CC BY · PubMed Central
Figure 11 Mapping of a CNN to TrueNorth. (A) Convolutional network features for one group at one topographic location are implemented using neurons on the same TrueNorth core, with their corresponding filter support region implemented using the core’s input lines, and filter weights implemented using the core’s synaptic array. The inputs are represented with white shapes, and the grey triangles represent the neurons. The filter used in each case is implemented mapping the matrix of weights (the numbers in the green boxes) into the synaptic array (grey circles); (B) For a neuron (blue points) to target multiple core inputs, its output (orange points) must be replicated by neuron copies, recruited from other neurons on the same core, or on extra cores if needed [76].
Figure 11 Mapping of a CNN to TrueNorth. (A) Convolutional network features for one group at one topographic location are implemented using neurons on the same TrueNorth core, with their corresponding filter support region implemented using the core’s input lines, and filter weights implemented using the core’s synaptic array. The inputs are represented with white shapes, and the grey triangles represent the neurons. The filter used in each case is implemented mapping the matrix of weights (the numbers in the green boxes) into the synaptic array (grey circles); (B) For a neuron (blue points) to target multiple core inputs, its output (orange points) must be replicated by neuron copies, recruited from other neurons on the same core, or on extra cores if needed [76].CC BY · PubMed Central
Figure 12 Sparse coding applied to audio. In red 20 basis functions learned from unlabeled audio, in blue the functions from cat auditory nerve fibers [113].
Figure 12 Sparse coding applied to audio. In red 20 basis functions learned from unlabeled audio, in blue the functions from cat auditory nerve fibers [113].CC BY · PubMed Central
Figure 13 Tripartite synapse represented by a presynaptic neuron, postsynaptic neuron and perisynaptic astrocyte (astrocyte process). The presynaptic neuron release neurotransmitters that are received by the postsynaptic neuron or the perisynaptic astrocyte [129].
Figure 13 Tripartite synapse represented by a presynaptic neuron, postsynaptic neuron and perisynaptic astrocyte (astrocyte process). The presynaptic neuron release neurotransmitters that are received by the postsynaptic neuron or the perisynaptic astrocyte [129].CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 127 references

Research works printed but not linked: 13

  1. Puri, M., Pathak, Y., Sutariya, V.K., Tipparaju, S., and Moreno, W. (2015). Artificial Neural Network for Drug Design, Delivery and Disposition, Elsevier Science.
  2. Lipton, Z.C. A Critical Review of Recurrent Neural Networks for Sequence Learning. Available online: http://arXiv Prepr arXiv1506.00019.
  3. Arenas, M.G., Mora, A.M., Romero, G., and Castillo, P.A. (2011). Advances in Computational Intelligence, Springer.
  4. Unterthiner, T., Mayr, A., Klambauer, G., Steijaert, M., Wegner, J.K., Ceulemans, H., and Hochreiter, S. (2014, January 8–13). Deep learning as an opportunity in virtual screening. Proceedings of the Deep Learning Workshop at NIPS, Montreal, QC, Canada.
  5. Dahl, G.E., Jaitly, N., and Salakhutdinov, R. Multi-Task Neural Networks for QSAR Predictions. Available online: http://arxiv.org/abs/1406.1231. · arXiv
  6. Pascanu, R., Gulcehre, C., Cho, K., and Bengio, Y. How to construct deep recurrent neural networks. Available online: http://arXiv Prepr arXiv1312.6026.
  7. Mead, C., and Conway, L. (1980). Introduction to VLSI Systems, Addison-Wesley.
  8. Dehaene, S. (2014). Consciousness and the Brain: Deciphering How the Brain Codes Our Thoughts, Viking Press.
  9. Wong, 2012, “1014” IBM Research Divsion, Research Report RJ10502, 2012, IBM J. Rep., 10502, 13
  10. Diehl, P.U., Pedroni, B.U., Cassidy, A., Merolla, P., Neftci, E., and Zarrella, G. TrueHappiness: Neuromorphic Emotion Recognition on Truenorth. Available online: http://arxiv.org/abs/1601.04183. · arXiv
  11. Serrano- Gotarredona, T., Linares-Barranco, B., Galluppi, F., Plana, L., and Furber, S. (2015). 2015 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE.
  12. Koob, A. (2009). The Root of Thought: Unlocking Glia—The Brain Cell That Will Help Us Sharpen Our Wits, Heal Injury, and Treat Brain Disease, FT Press.
  13. Porto, A., Pazos, A., and Araque, A. (2005). Computational Intelligence and Bioinspired Systems, Springer.

Cited by

Sorted by citations

See all 109

Related

Metadata

Authors
Pastur-Romay, L., Cedrón, F., Pazos, A., & Porto-Pazos, A.
Affiliations
Universidade da Coruña (ES)
Corresponding author
Porto-Pazos, Ana
ORCID
Venue
International Journal of Molecular Sciences
Publisher
MDPI AG
Published
2016-08-11
Volume
17
Issue
8
Pages
1313
ISSN
1422-0067 (ISSN-L)
DOI
10.3390/ijms17081313
PMC
PMC5000710
OpenAlex
W2517582793
Cited by
109
References
127
Citations by year
2026: 2 · 2025: 11 · 2024: 10 · 2023: 15 · 2022: 12 · 2021: 11 · 2020: 9 · 2019: 11 · 2018: 17 · 2017: 6 · 2016: 5
FWCI
1.00
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Molecular Sciences (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, cc-by)
  • RUC (Universidade Da Coruña) (submittedVersion, cc-by)
Funders
  • European Commission (2013-2016)
  • Xunta de Galicia (R2014/039, GRC2014/049)
  • Instituto de Salud Carlos III (2013-2016, PI13/00280)
  • Spanish National Plan for Scientific and Technical Research and Innovation
  • Center for Innovative Medicine
Topics
  • Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Drug Discovery › Computational Drug Discovery Methods
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
Keywords
deep neural network, neuromorphic hardware, drug design, virtual screening, QSAR, protein structure prediction, SpiNNaker, astrocytes, glial cells, bioinformatics
MeSH terms
Algorithms, Animals, Humans, Neural Networks, Computer, Computational Biology, Quantitative Structure-Activity Relationship

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review