CoChat
Sign Up
PreprintCited by 406Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningVisual Attention and Saliency Detection

Mask R-CNN

Open access
He, Kaiming, Gkioxari, Georgia, Dollár, Piotr, Girshick, Ross·2017·arXiv
DOI 10.48550/arxiv.1703.06870arXiv 1703.06870
Open this paper in CoChat — free accountOpen PDF

Abstract

We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R-CNN is simple to train and adds only a small overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to generalize to other tasks, e.g., allowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, including instance segmentation, bounding-box object detection, and person keypoint detection. Without bells and whistles, Mask R-CNN outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners. We hope our simple and effective approach will serve as a solid baseline and help ease future research in instance-level recognition. Code has been made available at: https://github.com/facebookresearch/Detectron

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1703.06870

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • FPN has enabled new top results in all tracks of the COCO competition, including detection, instance segmentation, and keypoint estimation.

    “Recently, FPN has enabled new top results in all tracks of the COCO competition, including detection, instance segmentation, and keypoint estimation. See [14] for details.”

    From Feature Pyramid Networks for Object Detection (2017) · cites this paper as [14]

    The paper reports strong COCO results using FPN backbones: its instance-segmentation model surpasses earlier winners (Section 4.1, p. 5), its detection model exceeds earlier single-model results (Section 4.3, p. 7), and its keypoint model exceeds the 2016 winner (Section 5, p. 8). But it does not establish that FPN enabled top competition results in all three tracks; Appendix B (p. 10) discusses 2017 winning teams only for instance segmentation and credits the Mask R-CNN framework. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 43

  1. [1][1] M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele. 2D human pose estimation: New benchmark and state of the art analysis. In CVPR , 2014. 8 -
  2. [2][2] P. Arbel´ aez, J. Pont-Tuset, J. T. Barron, F. Marques, and J. Malik. Multiscale combinatorial grouping. In CVPR , 2014. 2 -
  3. [3][3] A. Arnab and P. H. Torr. Pixelwise instance segmentation with a dynamically instantiated network. In CVPR , 2017. 3, 9 -
  4. [4][4] M. Bai and R. Urtasun. Deep watershed transform for instance segmentation. In CVPR , 2017. 3, 9 -
  5. [5][5] S. Bell, C. L. Zitnick, K. Bala, and R. Girshick. Insideoutside net: Detecting objects in context with skip pooling and recurrent neural networks. In CVPR , 2016. 5 -
  6. [6][6] Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh. Realtime multiperson 2d pose estimation using part affinity fields. In CVPR , 2017. 7, 8 -
  7. [7][7] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The Cityscapes dataset for semantic urban scene understanding. In CVPR , 2016. 9 -
  8. [8][8] J. Dai, K. He, Y. Li, S. Ren, and J. Sun. Instance-sensitive fully convolutional networks. In ECCV , 2016. 2 -
  9. [9][9] J. Dai, K. He, and J. Sun. Convolutional feature masking for joint object and stuff segmentation. In CVPR , 2015. 2 -
  10. [10][10] J. Dai, K. He, and J. Sun. Instance-aware semantic segmentation via multi-task network cascades. In CVPR , 2016. 2, 3, 4, 5, 6 -
  11. [11][11] J. Dai, Y . Li, K. He, and J. Sun. R-FCN: Object detection via region-based fully convolutional networks. In NIPS , 2016. 2 -
  12. [12][12] R. Girshick. Fast R-CNN. In ICCV , 2015. 1, 2, 3, 4, 6 -
  13. [13][13] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR , 2014. 2, 3 -
  14. [14][14] R. Girshick, F. Iandola, T. Darrell, and J. Malik. Deformable part models are convolutional neural networks. In CVPR , 2015. 4 -
  15. [15][15] B. Hariharan, P. Arbel´ aez, R. Girshick, and J. Malik. Simultaneous detection and segmentation. In ECCV . 2014. 2 -
  16. [16][16] B. Hariharan, P. Arbel´ aez, R. Girshick, and J. Malik. Hypercolumns for object segmentation and fine-grained localization. In CVPR , 2015. 2 -
  17. [17][17] Z. Hayder, X. He, and M. Salzmann. Shape-aware instance segmentation. In CVPR , 2017. 9 -
  18. [18][18] K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. In ECCV . 2014. 1, 2 -
  19. [19][19] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR , 2016. 2, 4, 7, 10 -
  20. [21][21] J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y . Song, S. Guadarrama, et al. Speed/accuracy trade-offs for modern convolutional object detectors. In CVPR , 2017. 2, 3, 4, 6, 7 -
  21. [22][22] M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu. Spatial transformer networks. In NIPS , 2015. 4 -
  22. [23][23] A. Kirillov, E. Levinkov, B. Andres, B. Savchynskyy, and C. Rother. Instancecut: from edges to instances with multicut. In CVPR , 2017. 3, 9 -
  23. [24][24] A. Krizhevsky, I. Sutskever, and G. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS , 2012. 2 -
  24. [26][26] Y. Li, H. Qi, J. Dai, X. Ji, and Y. Wei. Fully convolutional instance-aware semantic segmentation. In CVPR , 2017. 2, 3, 5, 6 -
  25. [27][27] T.-Y. Lin, P. Doll´ ar, R. Girshick, K. He, B. Hariharan, and S. Belongie. Feature pyramid networks for object detection. In CVPR , 2017. 2, 4, 5, 7 -
  26. [28][28] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ ar, and C. L. Zitnick. Microsoft COCO: Common objects in context. In ECCV , 2014. 2, 5 -
  27. [29][29] S. Liu, J. Jia, S. Fidler, and R. Urtasun. SGN: Sequential grouping networks for instance segmentation. In ICCV , 2017. 3, 9 -
  28. [30][30] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR , 2015. 1, 3, 6 -
  29. [31][31] V. Nair and G. E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML , 2010. 4 -
  30. [32][32] G. Papandreou, T. Zhu, N. Kanazawa, A. Toshev, J. Tompson, C. Bregler, and K. Murphy. Towards accurate multiperson pose estimation in the wild. In CVPR , 2017. 8 -
  31. [33][33] P. O. Pinheiro, R. Collobert, and P. Dollar. Learning to segment object candidates. In NIPS , 2015. 2, 3 -
  32. [34][34] P. O. Pinheiro, T.-Y. Lin, R. Collobert, and P. Doll´ ar. Learning to refine object segments. In ECCV , 2016. 2, 3 -
  33. [35][35] I. Radosavovic, P. Doll´ ar, R. Girshick, G. Gkioxari, and K. He. Data distillation: Towards omni-supervised learning. arXiv:1712.04440 , 2017. 10 - · arXiv
  34. [36][36] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS , 2015. 1, 2, 3, 4, 7 -
  35. [37][37] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In TPAMI , 2017. 10 -
  36. [38][38] A. Shrivastava, A. Gupta, and R. Girshick. Training regionbased object detectors with online hard example mining. In CVPR , 2016. 2, 5 -
  37. [39][39] A. Shrivastava, R. Sukthankar, J. Malik, and A. Gupta. Beyond skip connections: Top-down modulation for object detection. arXiv:1612.06851 , 2016. 4, 7 - · arXiv
  38. [40][40] C. Sun, A. Shrivastava, S. Singh, and A. Gupta. Revisiting unreasonable effectiveness of data in deep learning era. In ICCV , 2017. 10 -
  39. [41][41] C. Szegedy, S. Ioffe, and V. Vanhoucke. Inception-v4, inception-resnet and the impact of residual connections on learning. In ICLR Workshop , 2016. 7 -
  40. [42][42] J. R. Uijlings, K. E. van de Sande, T. Gevers, and A. W. Smeulders. Selective search for object recognition. IJCV , 2013. 2 -
  41. [43][43] X. Wang, R. Girshick, A. Gupta, and K. He. Non-local neural networks. arXiv:1711.07971 , 2017. 10 - · arXiv
  42. [44][44] S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh. Convolutional pose machines. In CVPR , 2016. 8 -
  43. [45][45] S. Xie, R. Girshick, P. Doll´ ar, Z. Tu, and K. He. Aggregated residual transformations for deep neural networks. In CVPR , 2017. 4, 10

Cited by

Sorted by citations

Authors
He, K., Gkioxari, G., Dollár, P., & Girshick, R.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2017
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1703.06870
arXiv
1703.06870
OpenAlex
W4394669457
Cited by
406
References
0
Citations by year
2026: 44 · 2025: 57 · 2024: 77 · 2023: 89 · 2022: 55 · 2021: 33 · 2020: 19 · 2019: 17 · 2018: 13 · 2017: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
Keywords
Mask R-CNN, instance segmentation, object detection, Faster R-CNN, segmentation masks, bounding box regression

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review