CoChat
Sign Up
Cited by 2,686Advanced Neural Network ApplicationsAdvanced Image and Video Retrieval TechniquesVisual Attention and Saliency Detection

Training Region-Based Object Detectors with Online Hard Example Mining

Open access
Shrivastava, Abhinav, Gupta, Abhinav, Girshick, Ross·2016·2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr.2016.89
Open this paper in CoChat — free accountOpen PDF

Abstract

The field of object detection has made significant advances riding on the wave of region-based ConvNets, but their training procedure still includes many heuristics and hyperparameters that are costly to tune. We present a simple yet surprisingly effective online hard example mining (OHEM) algorithm for training region-based ConvNet detectors. Our motivation is the same as it has always been - detection datasets contain an overwhelming number of easy examples and a small number of hard examples. Automatic selection of these hard examples can make training more effective and efficient. OHEM is a simple and intuitive algorithm that eliminates several heuristics and hyperparameters in common use. But more importantly, it yields consistent and significant boosts in detection performance on benchmarks like PASCAL VOC 2007 and 2012. Its effectiveness increases as datasets become larger and more difficult, as demonstrated by the results on the MS COCO dataset. Moreover, combined with complementary advances in the field, OHEM leads to state-of-the-art results of 78.9% and 76.3% mAP on PASCAL VOC 2007 and 2012 respectively.

Integrity signals

Registry

Crossref · 10.1109/cvpr.2016.89

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 11 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Using OHEM allows the foreground-background ratio hyperparameter to be dropped without harming performance.

    “With our proposed method, we can remove this ratio hyperparameter with no ill effect.”
    Quote in the paperSubstantial claim
  • Findingown data

    With VGGM, setting the background lower threshold to 0 instead of 0.1 lowers mAP by 2.4 points.

    “Table 1 (rows 3 -4) shows that for VGGM, mAP drops by 2.4 points, whereas for VGG16 it remains roughly the same.”
    Quote in the paperSubstantial claim
  • Findingown data

    OHEM improved AP for medium-sized objects by 4.9 points on the strict COCO AP metric.

    “OHEM helps improve the AP of medium sized objects by 4.9 points on the strict COCO AP evaluation metric”
    Quote in the paperNot a substantial claim
  • Findingown data

    With extra data on VOC12, OHEM reached 71.9% mAP versus Fast R-CNN's 68.4%, a 3.5-point gain.

    “With extra data, we achieve an mAP of 71.9% as compared to 68.4% mAP of FRCN, an improvement of 3.5 points.”
    Quote in the paperSubstantial claim
  • Findingown data

    With VGG16, setting the background lower threshold to 0 instead of 0.1 leaves mAP approximately unchanged.

    “Table 1 (rows 3 -4) shows that for VGGM, mAP drops by 2.4 points, whereas for VGG16 it remains roughly the same.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 27
  • The quote only partly states this claim. 6
  • Classified as probably not a substantial claim. 6

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    Increasing the mini-batch to 2048 and the learning rate, with gradient accumulation over two forward/backward passes, reduced training loss below the bg lo = 0.1 heuristic.

    “uses gradient accumulation over two forward/backward passes lowers the training loss below the bg lo = 0.1 heuristic.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    OHEM raised mAP to 75.1%, state-of-the-art for methods trained on VOC07 data.

    “But more importantly, using OHEM improves it to 75.1% mAP, which is state-ofthe-art for methods trained on VOC07 data (see Table 3).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Bottle, chair, and tvmonitor showed larger gains that held across the different PASCAL splits.

    “Bottle, chair, and tvmonitor show larger improvements that are consistent across the different PAS-CAL splits.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For a mini-batch of 2048, the best learning rates were 0.003 for VGG16 and 0.004 for VGGM.

    “We found optimal results by increasing it to 0.003 for VGG16 and 0.004 for VGGM.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    The heuristic that balances foreground and background RoIs at a 1:3 ratio is important when training FRCN.

    “We found that this is an important design decision for the training FRCN.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    OHEM produces lower training loss and higher test mAP.

    “we demonstrate that our approach leads to better training (lower training loss) and higher testing performance (mAP).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On VOC07, OHEM raised Fast R-CNN mAP from 67.2% to 69.9%, and with extra data from 70.0% to 74.6%.

    “Table 3 shows that on VOC07, OHEM improves the mAP of FRCN from 67.2% to 69.9% (and 70.0% to 74.6% with extra data).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    OHEM adds 0.09 seconds per training iteration for VGGM and 0.43 seconds for VGG16, and needs 1 GB more memory (2.3 GB for VGG16).

    “OHEM costs 0.09s per training iteration for VGGM network (0.43s for VGG16) and requires 1G more memory (2.3G for VGG16).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The background lower threshold of 0.1 aids convergence and detection accuracy.

    “We show in Section 5.4 that although this heuristic helps convergence and detection accuracy, it is suboptimal because it ignores some infrequent, but important, difficult background regions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Setting bg lo to 0 produced the highest training loss, while the heuristic bg lo = 0.1 produced a much lower loss.

    “We see that bg lo = 0 results in the highest training loss, while using the heuristic bg lo = 0.1 results in a much lower training loss.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    With that large batch and tuned learning rate, VGG16 and VGGM each gained about one mAP point over batch size 128, but the authors' method still exceeded training on all RoIs by more than one point.

    “Using these settings, mAP of both VGG16 and VGGM increased by ∼1 point compared to B = 128, but the improvement from our approach is still > 1 points over using all RoIs.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    A variant of the approach placed fourth overall in the 2015 MS COCO Detection Challenge.

    “In the 2015 MS COCO Detection Challenge, a variant of this approach finished 4 th place overall.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    The proposed two-RoI-network design uses about the same memory as the simple implementation but runs over twice as fast.

    “Under these settings, the proposed architecture (Figure 2) has similar memory footprint as the first option, but is > 2× faster.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingnone given

    Combining OHEM with multi-scale and iterative bounding-box regression gave state-of-the-art VOC results and competitive MS COCO results.

    “OHEM with the following two additions yields state-of-theart results on VOC and competitive results on MS COCO.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Multi-scale improved the method to 24.4% AP on the standard COCO metric and 44.4% AP50 on the VOC metric.

    “Using multi-scale improves the performance of our method to 24.4% AP on the standard COCO metric and to 44.4% AP 50 on the VOC metric.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    When used together with other recent improvements, OHEM reaches top reported mAP of 78.9% on PASCAL VOC 2007 and 76.3% on PASCAL VOC 2012.

    “Combined with these tricks, OHEM gives state-ofthe-art results of 78.9% and 76.3% mAP on PASCAL VOC 2007 and 2012, respectively.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [16]

      Not checked in the free run.

    • cites [13]

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    On the standard COCO metric, Fast R-CNN scored 19.7% AP and OHEM raised it to 22.6% AP.

    “On the standard COCO evaluation metric, FRCN [14] scores 19.7% AP, and OHEM improves it to 22.6% AP.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [14]

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    With one image per batch, the original Fast R-CNN's mAP fell by about one point, whereas the authors' training method kept mAP roughly unchanged.

    “As seen in Table 1 (rows 5 -6, 11), performance of the original FRCN drops by ∼1 point with N = 1, but when using our training procedure, mAP remains approximately the same.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    With extra data (07+12), OHEM reached 78.9% mAP, exceeding MR-CNN's 78.2% mAP.

    “When using extra data (07+12), OHEM achieves 78.9% mAP, surpassing the current stateof-the-art MR-CNN (78.2% mAP).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    OHEM leads to better training convergence and consistent detection-accuracy gains on standard benchmarks.

    “We conducted extensive experimental analysis to demonstrate the effectiveness of the proposed algorithm, which leads to better training convergence and consistent improvements in detection accuracy on standard benchmarks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On VOC07, Fast R-CNN with those additions reached 72.4% mAP and OHEM raised it to 75.1%, the highest reported score for that setting.

    “On VOC07, FRCN with the above mentioned additions achieves 72.4% mAP and OHEM improves it to 75.1%, which is currently the highest reported score under this setting (07 data).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    OHEM adapts the alternating hard-mining template to SGD by holding the model fixed for just one mini-batch, so updates occur at the same rate as baseline SGD and training is not slowed.

    “This strategy fits the alternation template to SGD by”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Both Fast R-CNN with and without OHEM were trained on the MS COCO training set.

    “Note that FRCN with and without OHEM were trained on MS COCO train set.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Interpretationnone given

    The gains from OHEM are orthogonal to multi-scale and iterative bounding-box regression.

    “In this section, we show that these improvements are orthogonal to recent bells and whistles that enhance object detection accuracy.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The results show that the heuristics are not optimal and that OHEM is effective.

    “This result demonstrates the sub-optimality of these heuristics and the effectiveness of our hard mining approach.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    The existing mini-batch construction method for SGD is inefficient and not optimal.

    “We argue that the current way of creating mini-batches for SGD (Section 3.1) is inefficient and suboptimal”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Highly overlapping RoIs tend to have correlated losses and can map to the same conv feature-map location, which double-counts their loss.

    “However, there is a small caveat: co-located RoIs with high overlap are likely to have correlated losses.Moreover, these overlapping RoIs can project onto the same region in the conv feature map, because of resolution disparity, thus leading to loss double counting.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The baseline results in Table 1 rows 1-2 were replicated with the authors' schedule and came out a little higher than in [14].

    “The baseline numbers reported in Table 1 (row 1-2) were reproduced using our training schedule and are slightly higher than the ones reported in [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14]

      Not checked in the free run.

  • The quote only partly states this claim.

    Interpretationown data

    The hard example mining approach is useful for smaller objects.

    “which indicates that the proposed hard example mining approach is helpful when dealing with smaller sized objects.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Choosing hard examples automatically can improve both the effectiveness and efficiency of training.

    “Automatic selection of these hard examples can make training more effective and efficient.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Detection datasets contain far more easy examples than hard examples.

  • May combine two claims.

    Interpretationreasoning

    OHEM removes several commonly used heuristics and hyperparameters by automatically choosing hard examples, which simplifies training.

    “OHEM eliminates several heuristics and hyperparameters in common use by automatically selecting hard examples, thus simplifying training.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    OHEM can train any region-based ConvNet detector, not only Fast R-CNN.

    “Though we used Fast R-CNN throughout this paper, OHEM can be used for training any region-based ConvNet detector.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Using a smaller mini-batch for gradient computation makes training faster.

    “Moreover, because we compute gradients with a smaller mini-batch size training is faster.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Forwarding all RoIs adds only modest computation, and the backward pass costs no more than before.

    “Most of the forward computation is shared between RoIs via the conv feature map, so the extra computation needed to for-ward all RoIs is relatively small.Moreover, because only a small number of RoIs are selected for updating the model, the backward pass is no more expensive than before.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Object detection training sets have a large imbalance between annotated object examples and background examples, up to 100,000:1 for sliding-window detectors and about 70:1 for proposal-based detectors.

    “In the case of sliding-window object detectors, such as the deformable parts model (DPM) [12], this imbalance may be as extreme as 100,000 background examples to every one object.The recent trend towards object-proposal-based detectors [15,32] mitigates this issue to an extent, but the imbalance ratio may still be high (e.g., 70:1).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [12]

      Not checked in the free run.

    • cites [15]

      Not checked in the free run.

    • cites [32] source

      Not checked in the free run.

35 of 35

  • The paper relies on this without support.

    Detection datasets contain far more easy examples than hard examples.

    “Our motivation is the same as it has always beendetection datasets contain an overwhelming number of easy examples and a small number of hard examples.”

  • The paper relies on this without support.

    Training deep ConvNet detectors with SGD usually needs hundreds of thousands of steps, and pausing the model for even a few iterations would greatly slow training.

    “Training deep ConvNet detectors with SGD typically requires hundreds of thousands of SGD steps and freezing the model for even a few iterations at a time would dramatically slow progress.”

  • The paper relies on this without support.

    Easy examples have low loss and contribute little to the gradient, so training automatically focuses on hard examples.

    “The easy examples will have low loss, and won't contribute much to the gradient; training will automatically focus on the hard examples.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Bootstrap was reintroduced to object detection after 2016.

    “To this end, the bootstrap was reintroduced to object detection after 2016 [23, 38, 101, 102].”

    From Object Detection in 20 Years: A Survey (2023) · cites this paper as [102]

    The Introduction describes this 2016 paper’s online hard example mining method as a novel bootstrapping technique for modern object detectors. But it says bootstrapping was already widely used in object detection; it does not support the claim that bootstrapping was reintroduced to the field after 2016. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • All recent top entries in the ImageNet and COCO detection challenges use multi-scale testing on featurized image pyramids.

    “All recent top entries in the ImageNet [33] and COCO [21] detection challenges use multi-scale testing on featurized image pyramids ( e.g ., [16, 35]).”

    From Feature Pyramid Networks for Object Detection (2017) · cites this paper as [35]

  • If exploited, image pyramids are used only at test time, creating an inconsistency between train/test-time inference.

    “and so, if exploited, image pyramids are used only at test time [15, 11, 16, 35], which creates an inconsistency between train/test-time inference.”

    From Feature Pyramid Networks for Object Detection (2017) · cites this paper as [35]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Shrivastava, A., Gupta, A., & Girshick, R.
Affiliations
  • Carnegie Mellon University (US)
  • Meta (United Kingdom) (GB)
ORCID
Venue
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2016-06
Pages
761-769
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr.2016.89
OpenAlex
W2341497066
Cited by
2,686
References
34
Citations by year
2026: 148 · 2025: 206 · 2024: 256 · 2023: 317 · 2022: 344 · 2021: 378 · 2020: 389 · 2019: 334 · 2018: 217 · 2017: 85 · 2016: 12
FWCI
74.92
Type
conference paper
Language
en
Open access
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Microsoft Research
  • Nvidia
  • Multidisciplinary University Research Initiative
  • Office of Naval Research (N00014-16-1-2007)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
Keywords
region-based object detection, online hard example mining, training efficiency, MS COCO

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review