Dynamic Routing Between Capsules
Open accessAbstract
A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or an object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule becomes active. We show that a discrimininatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognizing highly overlapping digits. To achieve these results we use an iterative routing-by-agreement mechanism: A lower-level capsule prefers to send its output to higher level capsules whose activity vectors have a big scalar product with the prediction coming from the lower-level capsule.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Because contextual memory contents are updated at inference time, recruiting some of them into the routing circuitry provides means to create new routes based on the first observations of a new sequence, suggesting an efficient alternative to capsule networks.
“Because the contents of contextual memory units are updated at inference time, recruiting some of them into the routing circuitry provides the means to create new routes on the basis of the first observations of a new sequence, suggesting an efficient alternative to capsule networks (Sabour et al., 2017).”
From Memory Mosaics (2024) · cites this paper as Sabour et al., 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408 · see citing works
Hinton, G. E., Krizhevsky, A., & Wang, S. D. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-21735-7_6 · Cited by 1,360 · see citing works
Self-citationLearning methods for generic object recognition with invariance to pose and lighting
LeCun, Y., Huang, F. J., & Bottou, L. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315150 · Cited by 1,324 · see citing works
Olshausen, B., Anderson, C., & Van Essen, D. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-11-04700.1993 · Cited by 908 · see citing works
Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
Goodfellow, I. J., Bulatov, Y., Ibarz, J., Arnoud, S., & Shet, V. (2013) · arXiv · doi:10.48550/arxiv.1312.6082 · Cited by 582 · see citing works
Flagged first, then by citations
Research works printed but not linked: 12
- Jimmy Ba, Volodymyr Mnih, and Koray Kavukcuoglu. Multiple object recognition with visual attention. arXiv preprint arXiv:1412.7755 , 2014. · arXiv
- Jia-Ren Chang and Yong-Sheng Chen. Batch-normalized maxout network in network. arXiv preprint arXiv:1511.02583 , 2015. · arXiv
- Dan C Cire¸ san, Ueli Meier, Jonathan Masci, Luca M Gambardella, and Jürgen Schmidhuber. Highperformance neural networks for visual object classification. arXiv preprint arXiv:1102.0183 , 2011. · arXiv
- Klaus Greff, Antti Rasmus, Mathias Berglund, Tele Hao, Harri Valpola, and Jürgen Schmidhuber. Tagger: Deep unsupervised perceptual grouping. In Advances in Neural Information Processing Systems , pages 4484-4492, 2016.
- Geoffrey E Hinton. Shape representation in parallel systems. In International Joint Conference on Artificial Intelligence Vol 2 , 1981a.
- Geoffrey E Hinton. A parallel computation that assigns canonical object-based frames of reference. In Proceedings of the 7th international joint conference on Artificial intelligence-Volume 2 , pages 683-685. Morgan Kaufmann Publishers Inc., 1981b.
- Geoffrey E Hinton, Zoubin Ghahramani, and Yee Whye Teh. Learning to parse images. In Advances in neural information processing systems , pages 463-469, 2000.
- Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial transformer networks. In Advances in Neural Information Processing Systems , pages 2017-2025, 2015.
- Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning , volume 2011, page 5, 2011.
- Denis G Pelli, Melanie Palomares, and Najib J Majaj. Crowding is unlike ordinary masking: Distinguishing feature integration from detection. Journal of vision , 4(12):12-12, 2004.
- Li Wan, Matthew D Zeiler, Sixin Zhang, Yann LeCun, and Rob Fergus. Regularization of neural networks using dropconnect. In Proceedings of the 30th International Conference on Machine Learning (ICML-13) , pages 1058-1066, 2013.
- Matthew D Zeiler and Rob Fergus. Stochastic pooling for regularization of deep convolutional neural networks. arXiv preprint arXiv:1301.3557 , 2013. · arXiv
Cited by
Sorted by citations
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906 · see citing works
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3084827 · Cited by 5,273 · see citing works
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846 · see citing works
“• Capsule [36] uses capsule structures [42] based on a VGG19 [44] network as the backbone architecture for DeepFake classification.”
MethodExplainable artificial intelligence (XAI) in deep learning-based medical image analysis
van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A., & Viergever, M. A. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102470 · Cited by 1,328 · see citing works
Lin, T., Wang, Y., Liu, X., & Qiu, X. (2022) · AI Open · doi:10.1016/j.aiopen.2022.10.001 · Cited by 1,422 · see citing works
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237 · see citing works
Deep convolutional neural network based medical image classification for disease diagnosis
Yadav, S. S., & Jadhav, S. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0276-2 · Cited by 1,069 · see citing works
Li, B., Li, Y., & Eliceiri, K. W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01409 · Cited by 957 · see citing works
Deep neural network models for computational histopathology: A survey
Srinidhi, C. L., Ciga, O., & Martel, A. L. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101813 · Cited by 749 · see citing works
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762 · see citing works
Knowledge Graph Embedding for Link Prediction
Rossi, A., Barbosa, D., Firmani, D., Matinata, A., & Merialdo, P. (2021) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3424672 · Cited by 601 · see citing works
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00083 · Cited by 600 · see citing works
Deep learning for deepfakes creation and detection: A survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103525 · Cited by 464 · see citing works
Deep Learning for Deepfakes Creation and Detection: A Survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2019) · arXiv · doi:10.48550/arxiv.1909.11573 · Cited by 464
Salient Object Detection via Integrity Learning
Zhuge, M., Fan, D.-P., Liu, N., Zhang, D., Xu, D., & Shao, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3179526 · Cited by 403 · see citing works
State-of-Charge Estimation of Li-Ion Battery in Electric Vehicles: A Deep Neural Network Approach
How, D. N. T., Hannan, M. A., Lipu, M. S. H., Sahari, K. S. M., Ker, P. J., & Muttaqi, K. M. (2020) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2020.3004294 · Cited by 337 · see citing works
Toraman, S., Alakus, T. B., & Turkoglu, I. (2020) · Chaos, Solitons & Fractals · doi:10.1016/j.chaos.2020.110122 · Cited by 327 · see citing works
How Much Position Information Do Convolutional Neural Networks Encode?
Islam, M. A., Jia, S., & Bruce, N. D. B. (2020) · arXiv · doi:10.48550/arxiv.2001.08248 · Cited by 253 · see citing works
Analog forecasting of extreme-causing weather patterns using deep learning
Chattopadhyay, A., Nabizadeh, E., & Hassanzadeh, P. (2020) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2019ms001958 · Cited by 224 · see citing works
Replication across space and time must be weak in the social and environmental sciences
Goodchild, M. F., & Li, W. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2015759118 · Cited by 157 · see citing works
Multimodal Generative Models for Scalable Weakly-Supervised Learning
Wu, M., & Goodman, N. (2018) · arXiv · doi:10.48550/arxiv.1802.05335 · Cited by 200 · see citing works
Samant, R. M., Bachute, M. R., Gite, S., & Kotecha, K. (2022) · IEEE Access · doi:10.1109/access.2022.3149798 · Cited by 123 · see citing works
Autonomous deep learning: A genetic DCNN designer for image classification
Ma, B., Li, X., Xia, Y., & Zhang, Y. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.10.007 · Cited by 120 · see citing works
Deepfake Attacks: Generation, Detection, Datasets, Challenges, and Research Directions
Naitali, A., Ridouani, M., Salahdine, F., & Kaabouch, N. (2023) · Computers · doi:10.3390/computers12100216 · Cited by 117 · see citing works
Gait Recognition Based on Deep Learning: A Survey
Filipi Gonçalves dos Santos, C., Oliveira, D. D. S., A. Passos, L., Gonçalves Pires, R., Felipe Silva Santos, D., Pascotti Valem, L., P. Moreira, T., Cleison S. Santana, M., Roder, M., Paulo Papa, J., et al. (2023) · ACM Computing Surveys · doi:10.1145/3490235 · Cited by 108 · see citing works
Kortylewski, A., Liu, Q., Wang, A., Sun, Y., & Yuille, A. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01401-3 · Cited by 98 · see citing works
DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection
Yan, Z., Zhang, Y., Yuan, X., Lyu, S., & Wu, B. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-0201 · Cited by 98 · see citing works
DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection
Yan, Z., Zhang, Y., Yuan, X., Lyu, S., & Wu, B. (2023) · arXiv · doi:10.48550/arxiv.2307.01426 · Cited by 97 · see citing works
“6) Capsule [29]: uses capsule structures [34] based on a VGG19 [38] network as the backbone architecture for deepfake classification.”
MethodA Comprehensive Survey on Passive Techniques for Digital Video Forgery Detection
Shelke, N. A., & Kasana, S. S. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09974-4 · Cited by 87 · see citing works
A Capsule Network Based Approach for Detection of Audio Spoofing Attacks
Luo, A., Li, E., Liu, Y., Kang, X., & Wang, Z. J. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9414670 · Cited by 84 · see citing works
Clune, J. (2019) · arXiv · doi:10.48550/arxiv.1905.10985 · Cited by 82 · see citing works
Wang, Z., Chen, C., Li, J., Wan, F., Sun, Y., & Wang, H. (2023) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2023.3237319 · Cited by 75 · see citing works
iCaps-Dfake: An Integrated Capsule-Based Model for Deepfake Image and Video Detection
Khalil, S. S., Youssef, S. M., & Saleh, S. N. (2021) · Future Internet · doi:10.3390/fi13040093 · Cited by 67 · see citing works
Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework ICCAD Special Session Paper
Shafique, M., Marchisio, A., Wicaksana Putra, R. V., & Hanif, M. A. (2021) · 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD) · doi:10.1109/iccad51958.2021.9643539 · Cited by 58 · see citing works
Associated Spatio-Temporal Capsule Network for Gait Recognition
Zhao, A., Dong, J., Li, J., Qi, L., & Zhou, H. (2021) · arXiv · doi:10.48550/arxiv.2101.02458 · Cited by 55 · see citing works
Assessment framework for deepfake detection in real-world situations
Lu, Y., & Ebrahimi, T. (2024) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-024-00621-8 · Cited by 60 · see citing works
Explainable Sentiment Analysis with Applications in Medicine
Zucco, C., Liang, H., Fatta, G. D., & Cannataro, M. (2018) · 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2018.8621359 · Cited by 59 · see citing works
Generative models for active vision
Parr, T., Sajid, N., Da Costa, L., Mirza, M. B., & Friston, K. J. (2021) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2021.651432 · Cited by 46 · see citing works
Scaife, A. M. M., & Porter, F. (2021) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stab530 · Cited by 44 · see citing works
Narayan, K., Agarwal, H., Thakral, K., Mittal, S., Vatsa, M., & Singh, R. (2022) · 2022 IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb54206.2022.10007968 · Cited by 26 · see citing works
A Survey on Deep Learning for Precision Oncology
Wang, C.-W., Khalil, M.-A., & Firdi, N. (2022) · Diagnostics · doi:10.3390/diagnostics12061489 · Cited by 23 · see citing works
ABC-CapsNet: Attention based Cascaded Capsule Network for Audio Deepfake Detection
Wani, T. M., Gulzar, R., & Amerini, I. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00253 · Cited by 14 · see citing works
Bae, J., Logan, P. E., Acri, D. J., Bharthur, A., Nho, K., Saykin, A. J., Risacher, S. L., Nudelman, K., Polsinelli, A. J., Pentchev, V., et al. (2023) · Alzheimer's & Dementia · doi:10.1002/alz.13319 · Cited by 14 · see citing works
Image Anomaly Detection with Capsule Networks and Imbalanced Datasets
Piciarelli, C., Mishra, P., & Foresti, G. L. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-30642-7_23 · Cited by 10 · see citing works
“Exercise difficulty prediction in online education systems,”
Fang, J., Zhao, W., & Jia, D. (2019) · 2019 International Conference on Data Mining Workshops (ICDMW) · doi:10.1109/icdmw.2019.00053 · Cited by 9 · see citing works
Zhang, J., Nolte, N., Sadhukhan, R., Chen, B., & Bottou, L. (2024) · arXiv · doi:10.48550/arxiv.2405.06394
“…the contents of contextual memory units are updated at inference time, recruiting some of them into the routing circuitry provides the means to create new routes on the basis of the first observations of a new sequence, suggesting an efficient alternative to capsule networks (Sabour et al., 2017).”
- Authors
- Sabour, S., Frosst, N., & Hinton, G. E.
- Affiliations
- Google (United States) (US)
- ORCID
- Sabour, Sara
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1710.09829
- OpenAlex
- W2751777443
- Cited by
- 4,167
- References
- 6
- Citations by year
- 2026: 12 · 2025: 215 · 2024: 395 · 2023: 492 · 2022: 518 · 2021: 848 · 2020: 727 · 2019: 653 · 2018: 300 · 2017: 6
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- capsule networks, dynamic routing, digit recognition, MNIST, discriminative training, object instantiation, transformation matrix, comparison with convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
