CoChat
Sign Up
Cited by 41Machine Learning and AlgorithmsAdvanced Neural Network ApplicationsAutonomous Vehicle Technology and Safety

Temporal Coherence for Active Learning in Videos

Open access
Zolfaghari Bengar, Javad, Gonzalez-Garcia, Abel, Villalonga, Gabriel, Raducanu, Bogdan, Habibi Aghdam, Hamed, Mozerov, Mikhail, Lopez, Antonio M., van de Weijer, Joost·2019·2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
DOI 10.1109/iccvw.2019.00120
Open this paper in CoChat — free accountOpen PDF

Abstract

Autonomous driving systems require huge amounts of data to train. Manual annotation of this data is time-consuming and prohibitively expensive since it involves human resources. Therefore, active learning emerged as an alternative to ease this effort and to make data annotation more manageable. In this paper, we introduce a novel active learning approach for object detection in videos by exploiting temporal coherence. Our active learning criterion is based on the estimated number of errors in terms of false positives and false negatives. The detections obtained by the object detector are used to define the nodes of a graph and tracked forward and backward to temporally link the nodes. Minimizing an energy function defined on this graphical model provides estimates of both false positives and false negatives. Additionally, we introduce a synthetic video dataset, called SYNTHIA-AL, specially designed to evaluate active learning for video object detection in road scenes. Finally, we show that our approach outperforms active learning baselines tested on two datasets.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/iccvw.2019.00120

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 52

  1. Y. Boykov and V. Kolmogorov. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. on PAMI, 26(9):1124-1137, 2004. 5
  2. W. Cai, Y. Zhang, S. Zhou, W. Wang, C. Ding, and X. Gu. Active learning for support vector machines with maximum model change. In Machine Learning and Knowledge Dis- covery in Databases, pages 211-226. Springer, 2014. 2
  3. K. Chitta, J. M. Alvarez, and A. Lesnikowski. Large-scale vi- sual active learning with deep probabilistic ensembles. arXiv preprint arXiv:1811.03575, 2018. 6 · arXiv
  4. M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, pages 3213-3223, 2016. 5
  5. I. Dagan and S. P. Engelson. Committee-based sampling for training probabilistic classifiers. In Machine Learning Pro- ceedings 1995, pages 150-157. Elsevier, 1995. 7
  6. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) chal- lenge. IJCV, 88(2):303-338, 2010. 3, 4, 5
  7. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ra- manan. Object detection with discriminatively trained part- based models. IEEE Trans. on PAMI, 32(9):1627-1645, 2010. 6
  8. A. Freytag, E. Rodner, and J. Denzler. Selecting influen- tial examples: Active learning with expected model output changes. In ECCV, pages 562-577, 2014. 2
  9. W. Fu, M. Wang, S. Hao, and X. Wu. Scalable active learning by approximated error reduction. In KDD, pages 1396-1405, 2018. 2
  10. E. Gavves, T. E. J. Mensink, T. Tommasi, and T. Snoek, C. G. M.and Tuytelaars. Active transfer learning with zero-shot priors: Reusing past datasets for future tasks. In ICCV, pages 1-9, 2015. 1, 2
  11. A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The kitti dataset. IJRR, 32(11):1231-1237, 2013. 1, 5
  12. Q. Gu, T. Z. Zhang, C. Ding, and J. Han. Selective labeling via error bound minimization. In NIPS, pages 1-9, 2012. 2
  13. Y. Guo. Active instance sampling via matrix partition. In NIPS, pages 1-9, 2010. 2
  14. W. Han, P. Khorrami, T. L. Paine, P. Ramachandran, M. Babaeizadeh, H. Shi, J. Li, S. Yan, and T. S. Huang. Seq-nms for video object detection. arXiv preprint arXiv:1602.08465, 2016. 2 · arXiv
  15. F. C. Heilbron, J.-Y. Lee, H. Jin, and B. Ghanem. What do i annotate next? an empirical study of active learning for action localization. In ECCV, pages 212-229, 2018. 2
  16. J. Hoffman, S. Guadarrama, E. Tzeng, J. Donahue, R. B. Gir- shick, T. Darrell, and K. Saenko. Lsda: Large scale detection through adaptation. In NIPS, pages 1-9, 2014. 2
  17. J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, et al. Speed/accuracy trade-offs for modern convolutional object detectors. In CVPR, pages 7310-7311, 2017. 6
  18. S.-J. Huang, R. Jin, and Z.-H. Zhou. Active learning by querying informative and representative examples. IEEE Trans. on PAMI, 10(36):1936-1949, 2014. 2
  19. S. Jin, A. RoyChowdhury, H. Jiang, A. Singh, A. Prasad, D. Chakraborty, and E. Learned-Miller. Unsupervised hard example mining from videos for improved object detection. In ECCV, pages 307-324, 2018. 1
  20. A. J. Joshi, F. Porikli, and N. P. Papanikolopoulos. Scal- able active learning for multiclass image classification. IEEE Trans. on PAMI, 34(11):2259-2273, 2012. 1, 2
  21. C. Käding, E. Rodner, A. Freytag, O. Mothes, B. Barz, and J. Denzler. Active learning for regression tasks with expected model output changes. In BMVC, pages 1-15, 2018. 2
  22. K. Kang, H. Li, T. Xiao, W. Ouyang, J. Yan, X. Liu, and X. Wang. Object detection in videos with tubelet proposal networks. In CVPR, pages 727-735, 2017. 1, 2
  23. K. Kang, H. Li, J. Yan, X. Zeng, B. Yang, T. Xiao, C. Zhang, Z. Wang, R. Wang, X. Wang, et al. T-cnn: Tubelets with con- volutional neural networks for object detection from videos. IEEE TCSVT, 28(10):2896-2907, 2018. 1, 2
  24. A. Kapoor, K. Grauman, R. Urtasun, and T. Darrell. Active learning with gaussian processes for object categorization. In CVPR, pages 1-8, 2007. 2
  25. V. Karasev, A. Ravichandran, and S. Soatto. Active frame, location, and detector selection for automated and manual video annotation. In CVPR, pages 2131-2138, 2014. 1
  26. V. Kolmogorov and R. Zabih. What energy functions can be minimizedvia graph cuts? IEEE Trans. on PAMI, 26(2):147- 159, 2004. 5
  27. D. D. Lewis and W. A. Gale. A sequential algorithm for training text classifiers. In SIGIR, pages 3-12, 1994. 6
  28. X. Li and Y. Guo. Adaptive active learning for image classi- fication. In cvpr, pages 860-866, 2013. 1
  29. X. Li and Y. Guo. Multi-level adaptive active learning for scene classification. In ECCV, pages 234-249, 2014. 1, 2
  30. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ra- manan, P. Dollár, and C. L. Zitnick. Microsoft coco: Com- mon objects in context. In ECCV, pages 740-755, 2014. 3, 6
  31. X. Lin and D. Parikh. Active learning for visual ques- tion answering: An empirical study. arXiv preprint arXiv:1711.01732, 2017. 2 · arXiv
  32. M. Liu and M. Zhu. Mobile video object detection with temporally-aware feature maps. In CVPR, pages 5686-5695, 2018. 2
  33. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg. Ssd: Single shot multibox detector. In ECCV, pages 21-37, 2016. 2
  34. V. Madhavan and T. Darrell. The bdd-nexar collective: A large-scale, crowsourced, dataset of driving scenes. Master's thesis, EECS Department, University of California, Berke- ley, May 2017. 1
  35. S. Ren, K. He, R. Girshick, and J. Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS, pages 91-99, 2015. 2, 4, 6
  36. A. Rosenfeld, R. Zemel, and J. K. Tsotsos. The elephant in the room. arXiv preprint arXiv:1808.03305, 2018. 1 · arXiv
  37. S. Roy, A. Unmesh, and V. P. Namboodiri. Deep active learn- ing for object detection. In BMVC, pages 1-12, 2018. 1, 2, 5, 6
  38. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. Imagenet large scale visual recog- nition challenge. IJCV, 115(3):211-252, 2015. 1, 6, 7, 8
  39. P. Saito, C. Suzuki, J. Gomes, P. de Rezende, and A. Falcão. Robust active learning for the diagnosis of parasites. Pattern Recognition, 48(11):3572-3583, 2015. 1, 2
  40. A. I. Schein and L. H. Ungar. Active learning for logistic regression: an evaluation. Machine Learning, 68(3):235- 265, 2007. 2
  41. O. Sener and S. Savarese. Active learning for convolutional neural networks: A core-set approach. In ICLR, pages 1-13, 2018. 2, 5, 7
  42. Stephan R. Richter, Z. Hayder, and V. Koltun. Playing for benchmarks. In ICCV, pages 2213-2222, 2017. 5
  43. A. Vezhnevets, J. M. Buhmann, and V. Ferrari. Active learn- ing for semantic segmentation with expected change. In CVPR, pages 3162-3169, 2012. 2
  44. S. Vijayanarasimhan and K. Grauman. Large-scale live ac- tive learning: Training object detectors with crawled data and crowds. IJCV, 108(1-2):97-114, 2014. 1, 2, 5
  45. S. Wang, Y. Zhou, J. Yan, and Z. Deng. Fully motion-aware network for video object detection. In ECCV, pages 542- 557, 2018. 1, 2
  46. Y. Yan, F. Nie, W. Li, C. Gao, Y. Yang, and D. Xu. Image classification by cross-media active learning with privileged information. IEEE Trans. on Multimedia, 18(12):2494- 2502, 2016. 1
  47. Y. Yang, Z. Ma, F. Nie, X. Chang, and A. G. Hauptmann. Multi-class active learning by uncertainty sampling with di- versity maximization. IJCV, 113(2):113-127, 2015. 2
  48. A. Yao, J. G. Gall, C. Leistner, and L. Van Gool. Interactive object detection. In CVPR, pages 3242-3249, 2012. 1, 2
  49. D. Yoo and I. S. Kweon. Learning loss for active learning. In CVPR, pages 93-102, 2019. 2, 5
  50. F. Yu, W. Xian, Y. Chen, F. Liu, M. Liao, V. Madha- van, and T. Darrell. Bdd100k: A diverse driving video database with scalable annotation tooling. arXiv preprint arXiv:1805.04687, 2018. 1, 5 · arXiv
  51. D. Zhang, F. Wang, Z. Shi, and C. Zhang. Interactive lo- calized content based image retrieval with multiple-instance active learning. Pattern Recognition, 43(2):478-484, 2010.
  52. X. Zhu, Y. Xiong, J. Dai, L. Yuan, and Y. Wei. Deep fea- ture flow for video recognition. In CVPR, pages 2349-2358, 2017. 1, 2

Cited by

Sorted by citations

Authors
Zolfaghari Bengar, J., Gonzalez-Garcia, A., Villalonga, G., Raducanu, B., Habibi Aghdam, H., Mozerov, M., Lopez, A. M., & van de Weijer, J.
Affiliations
Universitat Autònoma de Barcelona (ES)
ORCID
Venue
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
Publisher
IEEE
Published
2019-10
Pages
914-923
ISSN
2473-9936 (ISSN-L)
DOI
10.1109/iccvw.2019.00120
OpenAlex
W2970798461
Cited by
41
References
5
Citations by year
2026: 1 · 2025: 7 · 2024: 5 · 2023: 10 · 2022: 8 · 2021: 8 · 2020: 2
FWCI
1.72
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE International Conference on Computer Vision Workshops (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Centres de Recerca de Catalunya
  • Generalitat de Catalunya
  • Institució Catalana de Recerca i Estudis Avançats (TIN2017-88709-R)
  • CYTED Ciencia y Tecnología para el Desarrollo (518RT0559)
  • Agencia Estatal de Investigación
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
Keywords
active learning (machine learning), video object detection, temporal coherence, autonomous driving, data annotation, energy minimization, traffic scenes, synthetic video dataset

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review