Theano: new features and speed improvements
Open accessAbstract
Theano is a linear algebra compiler that optimizes a user's symbolically-specified mathematical computations to produce efficient low-level implementations. In this paper, we present new features and efficiency improvements to Theano, and benchmarks demonstrating Theano's performance relative to Torch7, a recently introduced machine learning library, and to RNNLM, a C++ library targeted at recurrent neural networks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Matplotlib: a 2D graphics environment
Hunter, J. D. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.55 · Cited by 39,249 · see citing works
Python for scientific computing
Oliphant, T. E. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.58 · Cited by 3,303 · cited 5× · see citing works
“Table 1 provides a summary comparison of the features provided by Torch7 (including the ones inherited from Lua) and Theano (including the ones coming from Python and NumPy/SciPy).”
MethodInfluential
Flagged first, then by citations
Research works printed but not linked: 7
- Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., and Bengio, Y. (2010). Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy) . Oral Presentation.
- Bergstra, J., Bastien, F., Breuleux, O., Lamblin, P., Pascanu, R., Delalleau, O., Desjardins, G., Warde-Farley, D., Goodfellow, I., Bergeron, A., and Bengio, Y. (2011). Theano: Deep learning on gpus with python. In Big Learn workshop, NIPS'11 .
- Collobert, R., Kavukcuoglu, K., and Farabet, C. (2011). Torch7: A matlab-like environment for machine learning. In BigLearn, NIPS Workshop .
- Martens, J. and Sutskever, I. (2011). Learning recurrent neural networks with hessian-free optimization. In L. Getoor and T. Scheffer, editors, Proceedings of the 28th International Conference on Machine Learning (ICML-11) , ICML '11, pages 1033-1040, New York, NY, USA. ACM.
- Mikolov, T., Deoras, A., Kombrink, S., Burget, L., and Cernocky, J. (2011). Empirical evaluation and combination of advanced language modeling techniques. In Proc. 12th annual conference of the international speech communication association (INTERSPEECH 2011) .
- P´ erez, F. and Granger, B. E. (2007). IPython: A system for interactive scientific computing. Computing in Science and Engineering , 9 (3), 21-29.
- Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning internal representations by error propagation. volume 1, chapter 8, pages 318-362. MIT Press, Cambridge.
Cited by
Sorted by citations
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Self-citationA survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332 · see citing works
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · see citing works
Self-citationExplaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
Self-citationShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
Self-citationEfficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation
Kamnitsas, K., Ledig, C., Newcombe, V. F. J., Simpson, J. P., Kane, A. D., Menon, D. K., Rueckert, D., & Glocker, B. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.10.004 · Cited by 3,498 · see citing works
Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images
Pereira, S., Pinto, A., Alves, V., & Silva, C. A. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2538465 · Cited by 2,735 · see citing works
Probabilistic programming in Python using PyMC3
Salvatier, J., Wiecki, T. V., & Fonnesbeck, C. (2016) · PeerJ Computer Science · doi:10.7717/peerj-cs.55 · Cited by 2,544 · see citing works
Attention-based LSTM for Aspect-level Sentiment Classification
Wang, Y., Huang, M., zhu, X., & Zhao, L. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1058 · Cited by 2,450 · see citing works
Lung pattern classification for interstitial lung diseases using a deep convolutional neural network
Anthimopoulos, M., Christodoulidis, S., Ebner, L., Christe, A., & Mougiakakou, S. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535865 · Cited by 1,343 · see citing works
Setio, A. A. A., Ciompi, F., Litjens, G., Gerke, P., Jacobs, C., van Riel, S. J., Wille, M. M. W., Naqibullah, M., Sanchez, C. I., & van Ginneken, B. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2536809 · Cited by 1,258 · see citing works
Describing Videos by Exploiting Temporal Structure
Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.512 · Cited by 1,035 · see citing works
Remaining useful life estimation of engineered systems using vanilla LSTM neural networks
Wu, Y., Yuan, M., Dong, S., Lin, L., & Liu, Y. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.05.063 · Cited by 789 · see citing works
Deep Learning in Medical Ultrasound Analysis: A Review
Liu, S., Wang, Y., Yang, X., Lei, B., Liu, L., Li, S. X., Ni, D., & Wang, T. (2019) · Engineering · doi:10.1016/j.eng.2018.11.020 · Cited by 798 · see citing works
Rotation-invariant convolutional neural networks for galaxy morphology prediction
Dieleman, S., Willett, K. W., & Dambre, J. (2015) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stv632 · Cited by 781 · see citing works
NiftyNet: a deep-learning platform for medical imaging
Gibson, E., Li, W., Sudre, C., Fidon, L., Shakir, D. I., Wang, G., Eaton-Rosen, Z., Gray, R., Doel, T., Hu, Y., et al. (2018) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.01.025 · Cited by 552 · see citing works
DeepCpG: accurate prediction of single-cell DNA methylation states using deep learning
Angermueller, C., Lee, H. J., Reik, W., & Stegle, O. (2017) · Genome Biology · doi:10.1186/s13059-017-1189-z · Cited by 536 · see citing works
Deep MRI brain extraction: A 3D convolutional neural network for skull stripping
Kleesiek, J., Urban, G., Hubert, A., Schwarz, D., Maier-Hein, K., Bendszus, M., & Biller, A. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.01.024 · Cited by 505 · see citing works
Convolutional embedding of attributed molecular graphs for physical property prediction
Coley, C. W., Barzilay, R., Green, W. H., Jaakkola, T. S., & Jensen, K. F. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00601 · Cited by 470 · see citing works
Brosch, T., Tang, L. Y. W., Yoo, Y., Li, D. K. B., Traboulsee, A., & Tam, R. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528821 · Cited by 460 · see citing works
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., & Chen, Z. (2020) · arXiv · doi:10.48550/arxiv.2006.16668 · Cited by 412 · see citing works
Atitallah, S. B., Driss, M., Boulila, W., & Ghézala, H. B. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100303 · Cited by 375 · see citing works
Song, H. F., Yang, G. R., & Wang, X.-J. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004792 · Cited by 281 · see citing works
Wolterink, J. M., Leiner, T., de Vos, B. D., van Hamersvelt, R. W., Viergever, M. A., & Išgum, I. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.04.004 · Cited by 321 · see citing works
Visualizing the Hidden Activity of Artificial Neural Networks
Rauber, P. E., Fadel, S. G., Falcao, A. X., & Telea, A. C. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2598838 · Cited by 280 · see citing works
Deep learning for galaxy surface brightness profile fitting
Tuccillo, D., Huertas-Company, M., Decencière, E., Velasco-Forero, S., Domínguez Sánchez, H., & Dimauro, P. (2018) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stx3186 · Cited by 93 · see citing works
GossipGraD: Scalable Deep Learning using Gossip Communication based Asynchronous Gradient Descent
Daily, J., Vishnu, A., Siegel, C., Warfel, T., & Amatya, V. (2018) · arXiv · doi:10.48550/arxiv.1803.05880 · Cited by 80 · see citing works
The cascading neural network: building the Internet of Smart Things
Leroux, S., Bohez, S., De Coninck, E., Verbelen, T., Vankeirsbilck, B., Simoens, P., & Dhoedt, B. (2017) · Knowledge and Information Systems · doi:10.1007/s10115-017-1029-1 · Cited by 59 · see citing works
Cheng, D., & Liu, M. (2017) · 2017 IEEE International Conference on Imaging Systems and Techniques (IST) · doi:10.1109/ist.2017.8261461 · Cited by 49 · see citing works
A Ubiquitous Machine Learning Accelerator with Automatic Parallelization on FPGA
Wang, C., Gong, L., Li, X., & Zhou, X. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.2990924 · Cited by 37 · see citing works
Representing high throughput expression profiles via perturbation barcodes reveals compound targets
Filzen, T. M., Kutchukian, P. S., Hermes, J. D., Li, J., & Tudor, M. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005335 · Cited by 31 · see citing works
Deep multimodal embedding: Manipulating novel objects with point-clouds, language and trajectories
Sung, J., Lenz, I., & Saxena, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989325 · Cited by 26 · see citing works
- Authors
- Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bouchard, N., Warde-Farley, D., & Bengio, Y.
- Affiliations
- Université de Montréal (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2012
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1211.5590
- OpenAlex
- W1606347560
- Cited by
- 1,005
- References
- 2
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 4 · 2023: 12 · 2022: 13 · 2021: 41 · 2020: 32 · 2019: 61 · 2018: 101 · 2017: 189 · 2016: 328 · 2015: 186 · 2014: 27 · 2013: 3 · 2012: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Compute Canada
- Canada Research Chairs
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Computational Physics and Python Applications
- Keywords
- Theano, symbolic computation, program optimization, performance benchmarking, recurrent neural networks, machine learning library
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
