The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
In-place Activated BatchNorm for Memory-Optimized Training of DNNs
Open accessAbstract
In this work we present In-Place Activated Batch Normalization (INPLACE-ABN) - a novel approach to drastically reduce the training memory footprint of modern deep neural networks in a computationally efficient way. Our solution substitutes the conventionally used succession of BatchNorm + Activation layers with a single plugin layer, hence avoiding invasive framework surgery while providing straightforward applicability for existing deep learning frameworks. We obtain memory savings of up to 50% by dropping intermediate results and by recovering required information during the backward pass through the inversion of stored forward results, with only minor increase (0.8-2%) in computation time. Also, we demonstrate how frequently used checkpointing approaches can be made computationally as efficient as INPLACE-ABN. In our experiments on image classification, we demonstrate on-par results on ImageNet-1k with state-of-the-art approaches. On the memory-demanding task of semantic segmentation, we report competitive results for COCO-Stuff and set new state-of-the-art results for Cityscapes and Mapillary Vistas. Code can be found at https://github.com/mapillary/inplace_abn.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Cited by
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00781 · Cited by 3,151
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer
Ranftl, R., Lasinger, K., Hafner, D., Schindler, K., & Koltun, V. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3019967 · Cited by 1,618
Panoptic Feature Pyramid Networks
Kirillov, A., Girshick, R., He, K., & Dollár, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00656 · Cited by 1,460
Object-Contextual Representations for Semantic Segmentation
Yuan, Y., Chen, X., & Wang, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58539-6_11 · Cited by 1,285
PointRend: Image Segmentation As Rendering
Kirillov, A., Wu, Y., He, K., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00982 · Cited by 1,135
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer
Ranftl, R., Lasinger, K., Hafner, D., Schindler, K., & Koltun, V. (2022) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000462024 · Cited by 1,093
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
Liu, C., Chen, L.-C., Schroff, F., Adam, H., Hua, W., Yuille, A. L., & Fei-Fei, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00017 · Cited by 1,025
Survey on semantic segmentation using deep learning techniques
Lateef, F., & Ruichek, Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.02.003 · Cited by 559
Disentangling Monocular 3D Object Detection
Simonelli, A., Bulo, S. R., Porzi, L., Lopez-Antequera, M., & Kontschieder, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00208 · Cited by 520
Self-citationSegmentation Transformer: Object-Contextual Representations for Semantic Segmentation
Yuan, Y., Chen, X., Chen, X., & Wang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11065 · Cited by 496
OCNet: Object Context Network for Scene Parsing
Yuan, Y., Huang, L., Guo, J., Zhang, C., Chen, X., & Wang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00916 · Cited by 453
Improving Semantic Segmentation via Video Propagation and Label Relaxation
Zhu, Y., Sapra, K., Reda, F. A., Shih, K. J., Newsam, S., Tao, A., & Catanzaro, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00906 · Cited by 411
IDD: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environments
Varma, G., Subramanian, A., Namboodiri, A., Chandraker, M., & Jawahar, C. V. (2019) · 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2019.00190 · Cited by 353
Modeling the Background for Incremental Learning in Semantic Segmentation
Cermelli, F., Mancini, M., Rota Bulo, S., Ricci, E., & Caputo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00925 · Cited by 340
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Wei, Y., Huang, L., Shi, H., Liu, W., & Huang, T. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.11721 · Cited by 338
ACFNet: Attentional Class Feature Network for Semantic Segmentation
Zhang, F., Chen, Y., Li, Z., Hong, Z., Liu, J., Ma, F., Han, J., & Ding, E. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00690 · Cited by 315
Improving Semantic Segmentation via Decoupled Body and Edge Supervision
Li, X., Li, X., Zhang, L., Cheng, G., Shi, J., Lin, Z., Tan, S., & Tong, Y. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58520-4_26 · Cited by 304
Self-supervised model adaptation for multimodal semantic segmentation
Valada, A., Mohan, R., & Burgard, W. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01188-y · Cited by 293
Boundary-Aware Feature Propagation for Scene Segmentation
Ding, H., Jiang, X., Liu, A. Q., Thalmann, N. M., & Wang, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00692 · Cited by 271
OCNet: Object Context for Semantic Segmentation
Yuan, Y., Huang, L., Guo, J., Zhang, C., Chen, X., & Wang, J. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01465-9 · Cited by 259
Johnston, A., & Carneiro, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00481 · Cited by 255
Machine Learning for Microcontroller-Class Hardware: A Review
Saha, S. S., Sandha, S. S., & Srivastava, M. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3210773 · Cited by 253
Online Model Distillation for Efficient Video Inference
Mullapudi, R. T., Chen, S., Zhang, K., Ramanan, D., & Fatahalian, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00367 · Cited by 96
Learning Multi-Object Tracking and Segmentation From Automatic Annotations
Porzi, L., Hofinger, M., Ruiz, I., Serrat, J., Bulo, S. R., & Kontschieder, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00688 · Cited by 82
Self-citationLearning Deep Multimodal Feature Representation with Asymmetric Multi-layer Fusion
Wang, Y., Sun, F., Lu, M., & Yao, A. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413621 · Cited by 68
3D Scene Reconstruction With Multi-Layer Depth and Epipolar Transformers
Shin, D., Ren, Z., Sudderth, E., & Fowlkes, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00226 · Cited by 57
Pedestrian detection in automotive safety: Understanding state-of-the-art
Ragesh, N. K., & Rajesh, R. (2019) · IEEE Access · doi:10.1109/access.2019.2909992 · Cited by 57
Advisable Learning for Self-Driving Vehicles by Internalizing Observation-to-Action Rules
Kim, J., Moon, S., Rohrbach, A., Darrell, T., & Canny, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00968 · Cited by 45
Metadata
- Authors
- Bulo, S. R., Porzi, L., & Kontschieder, P.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 5639-5647
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2772805599
- Cited by
- 320
- References
- 38
- FWCI
- 16.77
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.