Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 367 of the 367 references it cites were found in a registry or an index; 17 of the 301 sources its claims cite were checked for support.

Deep Learning for Generic Object Detection: A Survey

Open access
Liu, Li, Ouyang, Wanli, Wang, Xiaogang, Fieguth, Paul, Chen, Jie, Liu, Xinwang, Pietikäinen, Matti · 2020 · International Journal of Computer Vision

Abstract

Object detection, one of the most fundamental and challenging problems in computer vision, seeks to locate object instances from a large number of predefined categories in natural images. Deep learning techniques have emerged as a powerful strategy for learning feature representations directly from data and have led to remarkable breakthroughs in the field of generic object detection. Given this period of rapid evolution, the goal of this paper is to provide a comprehensive survey of the recent achievements in this field brought about by deep learning techniques. More than 300 research contributions are included in this survey, covering many aspects of generic object detection: detection frameworks, object feature representation, object proposal generation, context modeling, training strategies, and evaluation metrics. We finish the survey by identifying promising directions for future research.

Overview

CoChat Summary

TL;DR

This survey reviews deep learning for generic object detection, covering more than 300 research contributions across detection frameworks, feature representation, proposal generation, context modeling, training strategies, and evaluation metrics. It organizes the field into two-stage and one-stage detectors, summarizes datasets and benchmarks, and identifies open challenges such as small-object detection, context reasoning, weak/few-shot learning, and open-world detection.

Key findings

  • The survey provides a taxonomy of deep learning based generic object detection, including region-based/two-stage frameworks (RCNN, Fast RCNN, Faster RCNN, RFCN, Mask RCNN) and unified/one-stage frameworks (YOLO, SSD), plus backbone CNN architectures, multiscale feature fusion, context modeling, detection proposals, and training strategies.
  • Two-stage detectors generally achieve higher accuracy when large computation is allowed, while one-stage detectors are generally faster but weaker on small objects; for example, SSD achieves 74.3% mAP on VOC2007 test at 59 FPS versus Faster RCNN at 7 FPS / 73.2% mAP and YOLO at 45 FPS / 63.4% mAP. YOLO runs at 45 FPS and Fast YOLO at 155 FPS.
  • The main datasets are PASCAL VOC, ImageNet/ILSVRC, MS COCO, and Open Images; COCO contains 41% small and 24% large objects, and the best COCO detector trained on 80 classes reaches only 73% mAP even at 0.5 IoU, while ImageNet Top-5 classification error dropped from 16% to 2% in less than five years. The Open Images challenge winner achieved 61.71% mAP on the public leaderboard and 58.66% mAP on the private leaderboard.
  • Future directions include open-world learning, better and more efficient detection frameworks, compact CNN features, automatic neural architecture search, object instance segmentation, weakly supervised detection, few/zero-shot detection, other modalities, and universal object detection.

Methods

  • Design: a comprehensive literature survey of deep learning based generic object detection, focusing on major progress of the last 5 years, restricted to still pictures and to top journal and conference papers, with more than 300 research contributions included.
  • Organization: the survey presents a taxonomy covering detection frameworks, object feature representation, object proposal generation, context modeling, training strategies, and evaluation metrics, and summarizes popular datasets and evaluation criteria such as AP/mAP and IOU thresholds.
  • Analysis: no new experiments are reported; the paper synthesizes and compares reported results across PASCAL VOC, ILSVRC, MS COCO, and Open Images, and discusses representative methods in tables and figures.

Limitations

  • Stated: the survey limits its focus to top journal and conference papers, covers mainly the last 5 years, restricts attention to still pictures, and leaves video object detection for separate future consideration; the authors apologize to authors whose works are not included.
  • Stated: comparing detectors by their originally reported performance may be misleading because methods differ in meta-frameworks, backbone networks, pretraining datasets, proposal methods, augmentation, and ensembling; there has been very limited unified comparison work.
  • Evident: the survey is not exhaustive given the breadth of the field, and it excludes extensive work on specific object detection such as face, pedestrian, vehicle, and traffic sign detection; the provided text also contains fragmented tables and figures, which limits some detail.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s11263-019-01247-4

Cites

367 of 367 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Image 1
CC-BY · From the PDF
Image 2
CC-BY · From the PDF
Image 3
CC-BY · From the PDF
My neigbour's dog
My neigbour's dogCC-BY · From the PDF
Fig. 2 Object detection includes localizing instances of a particular object (top), as well as generalizing to detecting object categories in general (bottom). This survey focuses on recent advances for the latter problem of generic object detection
Fig. 2 Object detection includes localizing instances of a particular object (top), as well as generalizing to detecting object categories in general (bottom). This survey focuses on recent advances for the latter problem of generic object detectionCC-BY · From the PDF
Fig. 3 An overview of recent object detection performance: we can observe a significant improvement in performance (measured as mean average precision) since the arrival of deep learning in 2012. a Detection results of winning entries in the VOC2007-2012 competitions, and b top object detection competition results in ILSVRC2013-2017 (results in both panels use only the provided training data)
Fig. 3 An overview of recent object detection performance: we can observe a significant improvement in performance (measured as mean average precision) since the arrival of deep learning in 2012. a Detection results of winning entries in the VOC2007-2012 competitions, and b top object detection competition results in ILSVRC2013-2017 (results in both panels use only the provided training data)CC-BY · From the PDF
Fig. 4 Recognition problems related to generic object detection: a image level object classification, b bounding box level generic object detection, c pixel-wise semantic segmentation, d instance level semantic segmentation
Fig. 4 Recognition problems related to generic object detection: a image level object classification, b bounding box level generic object detection, c pixel-wise semantic segmentation, d instance level semantic segmentationCC-BY · From the PDF
Fig. 5 Taxonomy of challenges in generic object detection
Fig. 5 Taxonomy of challenges in generic object detectionCC-BY · From the PDF
Image 9
CC-BY · From the PDF
Image 10
CC-BY · From the PDF
Fig. 7 Milestones of object detection and recognition, including feature representations (Csurka et al. 2004; Dalal and Triggs 2005; He et al. 2016; Krizhevsky et al. 2012a; Lazebnik et al. 2006; Lowe 1999, 2004; Perronnin et al. 2010; Simonyan and Zisserman 2015; Sivic and Zisserman2003;Szegedyetal. 2015; Viola and Jones 2001; Wang et al. 2009), detection frameworks (Felzenszwalb et al. 2010b; Girshick et al. 2014; Sermanet et al. 2014; Uijlings et al. 2013; Viola and Jones 2001), and
Fig. 7 Milestones of object detection and recognition, including feature representations (Csurka et al. 2004; Dalal and Triggs 2005; He et al. 2016; Krizhevsky et al. 2012a; Lazebnik et al. 2006; Lowe 1999, 2004; Perronnin et al. 2010; Simonyan and Zisserman 2015; Sivic and Zisserman2003;Szegedyetal. 2015; Viola and Jones 2001; Wang et al. 2009), detection frameworks (Felzenszwalb et al. 2010b; Girshick et al. 2014; Sermanet et al. 2014; Uijlings et al. 2013; Viola and Jones 2001), andCC-BY · From the PDF
Fig. 8 a Illustration of three operations that are repeatedly applied by a typical CNN: convolution with a number of linear filters; Nonlinearities (e.g. ReLU); and local pooling (e.g. max pooling). The M feature maps from a previous layer are convolved with N different filters (here shown as size 3 × 3 × M ), using a stride of 1. The resulting N feature maps are then passed through a nonlinear function (e.g. ReLU), and pooled (e.g. taking a maximum over 2 × 2 regions) to give N feature maps at a reduced resolution. b Illustration of the architecture of VGGNet (Simonyan and Zisserman 2015), a typical CNN with 11 weight layers.
Fig. 8 a Illustration of three operations that are repeatedly applied by a typical CNN: convolution with a number of linear filters; Nonlinearities (e.g. ReLU); and local pooling (e.g. max pooling). The M feature maps from a previous layer are convolved with N different filters (here shown as size 3 × 3 × M ), using a stride of 1. The resulting N feature maps are then passed through a nonlinear function (e.g. ReLU), and pooled (e.g. taking a maximum over 2 × 2 regions) to give N feature maps at a reduced resolution. b Illustration of the architecture of VGGNet (Simonyan and Zisserman 2015), a typical CNN with 11 weight layers.CC-BY · From the PDF
An image with 3 color channels is presented as the input. The network has 8 convolutional layers, 3 fully connected layers, 5 max pooling layers and a softmax classification layer. The last three fully connected layers take features from the top convolutional layer as input in vector form. The final layer is a C -way softmax function, C being the number of classes. The whole network can be learned from labeled training data by optimizing an objective function (e.g. mean squared error or cross entropy loss) via stochastic gradient descent (Color figure online)
An image with 3 color channels is presented as the input. The network has 8 convolutional layers, 3 fully connected layers, 5 max pooling layers and a softmax classification layer. The last three fully connected layers take features from the top convolutional layer as input in vector form. The final layer is a C -way softmax function, C being the number of classes. The whole network can be learned from labeled training data by optimizing an objective function (e.g. mean squared error or cross entropy loss) via stochastic gradient descent (Color figure online)CC-BY · From the PDF
Image 14
CC-BY · From the PDF
Table 4 Most frequent object classes for each detection challenge
Table 4 Most frequent object classes for each detection challengeCC-BY · From the PDF
Fig. 11 Milestones in generic object detection
Fig. 11 Milestones in generic object detectionCC-BY · From the PDF
Fig. 12 Illustration of the RCNN detection framework (Girshick et al. 2014, 2016)
Fig. 12 Illustration of the RCNN detection framework (Girshick et al. 2014, 2016)CC-BY · From the PDF
Image 18
CC-BY · From the PDF
Fig. 14 Illustration of the OverFeat (Sermanet et al. 2014) detection framework
Fig. 14 Illustration of the OverFeat (Sermanet et al. 2014) detection frameworkCC-BY · From the PDF
Fig. 15 Performance of winning entries in the ILSVRC competitions from 2011 to 2017 in the image classification task
Fig. 15 Performance of winning entries in the ILSVRC competitions from 2011 to 2017 in the image classification taskCC-BY · From the PDF
Fig. 16 Comparison of HyperNet and ION. LRN is local response normalization, which performs a kind of 'lateral inhibition' by normalizing over local input regions (Jia et al. 2014)
Fig. 16 Comparison of HyperNet and ION. LRN is local response normalization, which performs a kind of 'lateral inhibition' by normalizing over local input regions (Jia et al. 2014)CC-BY · From the PDF
Fig. 17 Hourglass architectures: Conv1 to Conv5 are the main Conv blocks in backbone networks such as VGG or ResNet. The figure compares a number of feature fusion blocks (FFB) commonly used in recent approaches: FPN (Lin et al. 2017a), TDM (Shrivastava et al. 2017),
Fig. 17 Hourglass architectures: Conv1 to Conv5 are the main Conv blocks in backbone networks such as VGG or ResNet. The figure compares a number of feature fusion blocks (FFB) commonly used in recent approaches: FPN (Lin et al. 2017a), TDM (Shrivastava et al. 2017),CC-BY · From the PDF
Fig. 18 Representative approaches that explore local surrounding contextual features: MRCNN (Gidaris and Komodakis 2015), GBDNet (Zeng et al. 2016, 2017), ACCNN (Li et al. 2017b) and CoupleNet (Zhu et al. 2017a); also see Table 8
Fig. 18 Representative approaches that explore local surrounding contextual features: MRCNN (Gidaris and Komodakis 2015), GBDNet (Zeng et al. 2016, 2017), ACCNN (Li et al. 2017b) and CoupleNet (Zhu et al. 2017a); also see Table 8CC-BY · From the PDF
Fig. 19 Illustration of the region proposal network (RPN) introduced in Ren et al. (2015)
Fig. 19 Illustration of the region proposal network (RPN) introduced in Ren et al. (2015)CC-BY · From the PDF
Fig. 20 Localization error could stem from insufficient overlap or duplicate detections. Localization error is a frequent cause of false positives (Color figure online)
Fig. 20 Localization error could stem from insufficient overlap or duplicate detections. Localization error is a frequent cause of false positives (Color figure online)CC-BY · From the PDF
Fig. 21 Evolution of object detection performance on COCO (Test-Dev results). Results are quoted from (Girshick 2015; He et al. 2017; Ren et al. 2017). The backbone network, the design of detection framework and the availability of good and large scale datasets are the three most important factors in detection accuracy
Fig. 21 Evolution of object detection performance on COCO (Test-Dev results). Results are quoted from (Girshick 2015; He et al. 2017; Ren et al. 2017). The backbone network, the design of detection framework and the availability of good and large scale datasets are the three most important factors in detection accuracyCC-BY · From the PDF

Verification

Claims the paper makes

14 Valid · 16 Not valid · 154 Not checked · 108 Own results and interpretations

98 more claims · 5 with their cited sources checked

Create a free account to see all 292 claims and to verify the 284 references still open.

See all claims — free account
  • Probably not a substantial claim. 108
  • It may state two separate claims as one. 121
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

2 of 2

Citations

Cites

  • Deep Residual Learning for Image Recognition

    He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×

    “GNet. Similarly, ResNet demonstrated the effectiveness of skip connections for learning extremely deep networks with hundreds of layers, winning the ILSVRC 2015 classification task. Inspired by ResNet [101], InceptionResNets [265] combined the Inception networks with shortcut connections, on the basis that shortcut con…”
    BackgroundMethodInfluential
  • Attention Is All You Need

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×

    “ning architecture where image and memory are processed in parallel to obtain detections which further update memory. Inspired by the recent success of attention modules in natural language processing [274], ORN processes a set of objects simultaneously through the interaction between their appearance 22 Li Liu et al.…”
    Background
  • Very Deep Convolutional Networks for Large-Scale Image Recognition

    Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×

    “including the following choices: Meta detection frameworks, such as RCNN [85], Fast RCNN [84], Faster RCNN [229], RFCN [50], Mask RCNN [102], YOLO [227] and SSD [175]; Backbone networks such as VGG [248], Inception [263,125, 264], ResNet [101], ResNeXt [291], and Xception [45] etc. listed in Table6; Innovations such a…”
    Method
  • ImageNet classification with deep convolutional neural networks

    Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155

  • Long Short-Term Memory

    Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×

    “Noting that message passing is not always helpful but dependent on individual samples, Zeng et al. used gated functions to control message transmission, like in Long Short Term Memory (LSTM) networks [83]. Concurrent with GBDNet, Li et al. [123] presented ACCNN (Fig.13(c)) to utilize both global and local contextual i…”
    Method
  • Deep learning

    LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126

  • Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

    Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 5×

    “Ren ら [229, 230] により提案された Faster RCNN フ レームワークは,物体領域候補(領域提案)生成用に効率的で Input Image Region Proposals Extract CNN Features Each Warped Region”
    Influential
  • Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

    Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 5×

    “Ren ら [229, 230] により提案された Faster RCNN フ レームワークは,物体領域候補(領域提案)生成用に効率的で Input Image Region Proposals Extract CNN Features Each Warped Region”
    Influential
  • ImageNet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 4×

    “pretrained network is typically finetuned11 on a given detection dataset [58,83,85]. Several large scale image classification datasets are used for CNN pretraining. Among them the ImageNet1000 dataset [52,230] with 1.2 million images of 1000 object categories, or the Places dataset [311] which is much larger than ImageN…”
    BackgroundMethodInfluential
  • Gradient-based learning applied to document recognition

    Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805

Show all 349 references

Research works printed but not linked: 7

  1. Biederman, I. (1987b). Recognition by components: A theory of human image understanding. Psychological Review, 94(2), 115. · DOI · Paper page
  2. Chen, G., Choi, W., Yu, X., Han, T., & Chandraker M. (2017a). Learning efficient object detection models with knowledge distillation. In NIPS.
  3. Krizhevsky, A., Sutskever, I., & Hinton, G. (2012b). ImageNet classification with deep convolutional neural networks. In NIPS (pp. 1097–1105).
  4. Murase, H., & Nayar, S. (1995b). Visual learning and recognition of 3d objects from appearance. IJCV, 14(1), 5–24. · DOI · Paper page
  5. Ren, M., Triantafillou, E., Ravi, S., Snell, J., Swersky, K., Tenenbaum, J. B., Larochelle, H., & Zemel R. S. (2018). Meta learning for semisupervised few shot classification. In ICLR.
  6. Wan, L., Eigen, D., & Fergus, R. (2015). End to end integration of a convolution network, deformable parts model and nonmaximum suppression. In CVPR (pp. 851–859).
  7. Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014). How transferable are features in deep neural networks? In NIPS (pp. 3320–3328).

Cited by

See all 2,965

Metadata

Authors
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M.
Affiliations
  • National University of Defense Technology (CN)
  • The University of Sydney (AU)
  • Chinese University of Hong Kong (HK)
  • University of Waterloo (CA)
  • University of Oulu (FI)
Corresponding author
Liu, Li
Venue
International Journal of Computer Vision
Published
2019-10-31
Volume
128
Issue
2
Pages
261-318
ISSN
  • 0920-5691 (ISSN-L)
  • 1573-1405
OpenAlex
W2988916019
Cited by
2,965
References
349
Citations by year
2026: 217 · 2025: 362 · 2024: 396 · 2023: 485 · 2022: 538 · 2021: 544 · 2020: 319 · 2019: 92
FWCI
166.93
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Computer Vision (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
NSFC China (61872379)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
Keywords
generic object detection, instance segmentation, object proposal generation, context modeling, training strategies, evaluation metrics, detection framework, deep learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (prompt extract-v3).

A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Deep Learning for Generic Object Detection: A Survey. https://app.cochat.ai/papers/10.1007/s11263-019-01247-4
The paper
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2020). Deep Learning for Generic Object Detection: A Survey. International Journal of Computer Vision. https://doi.org/10.1007/s11263-019-01247-4

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.