Learning visual feature descriptors for dynamic lighting conditions
The DOI matches its Crossref record; 41 of the 42 references it cites were found in a registry or an index.
Abstract
In many robotic applications, especially long-term outdoor deployments, the success or failure of feature-based image registration is largely determined by changes in lighting. This paper reports on a method to learn visual feature point descriptors that are more robust to changes in scene lighting than standard hand-designed features. We demonstrate that, by tracking feature points in time-lapse videos, one can easily generate training data that captures how the visual appearance of interest points changes with lighting over time. This training data is used to learn feature descriptors that map the image patches associated with feature points to a lower-dimensional feature space where Euclidean distance provides good discrimination between matching and non-matching image patches. Results showing that the learned descriptors increase the ability to register images under varying lighting conditions are presented for a challenging indoor-outdoor dataset spanning 27 mapping sessions over a period of 15 months, containing a wide variety of lighting changes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
(2010) · International Conference on Machine Learning · Cited by 13,995
Speeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557 · cited 17×
“The dashed green lines show the matching cost function using similarity weighting (2) with α = 1/8 h for |ti − tj | = [2, 4, 5, 6, 10] h.”
BackgroundMethodInfluentialVideo Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367 · cited 1×
“It is worth noting that the feature descriptors learned using our proposed method could be used in a bag-of-words model [11, 12] for place recognition, however, evaluating if this would improve the robustness with respect to lighting remains future work.”
MethodDimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
Learning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036
Scalable Recognition with a Vocabulary Tree
Nister, D., & Stewenius, H. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.264 · Cited by 3,579
Multiple View Geometry in Computer Vision (2nd ed)
Hartley, R. I., & Zisserman, A. (2003) · Cambridge University Press eBooks · Cited by 2,761
Neighbourhood Components Analysis
Goldberger, J., Hinton, G. E., Roweis, S. T., & Salakhutdinov, R. (2004) · neural information processing systems · Cited by 2,424
Signature Verification Using A "Siamese" Time Delay Neural Network
BROMLEY, J., BENTZ, J. W., BOTTOU, L., GUYON, I., LECUN, Y., MOORE, C., SÄCKINGER, E., & SHAH, R. (1993) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001493000339 · Cited by 2,077
(2002) · Cited by 1,635
DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo
Tola, E., Lepetit, V., & Fua, P. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.77 · Cited by 1,405 · cited 10×
“An example image pair that was successfully registered using the CMLP descriptor, but not SIFT, SURF, nor DAISY is shown in Fig.”
BackgroundMethodInfluentialSalakhutdinov, R., & Hinton, G. (2009) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2008.11.006 · Cited by 1,246 · cited 1×
“Beyond Siamese networks, auto-encoder frameworks can also be used to learn nonlinear embeddings as shown in Hinton and Salakhutdinov [29] and [30].”
MethodTheano: A CPU and GPU Math Compiler in Python
Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., & Bengio, Y. (2010) · Proceedings of the Python in Science Conference · doi:10.25080/majora-92bf1922-003 · Cited by 1,052
Fast Keypoint Recognition Using Random Ferns
Ozuysal, M., Calonder, M., Lepetit, V., & Fua, P. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.23 · Cited by 595
Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure
Salakhutdinov, R., & Hinton, G. E. (2007) · International Conference on Artificial Intelligence and Statistics · Cited by 586
Discriminative Learning of Local Image Descriptors
Brown, M., Hua, G., & Winder, S. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.54 · Cited by 466
Learning Local Image Descriptors
Winder, S. A. J., & Brown, M. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.382971 · Cited by 339
Deep learning from temporal coherence in video
Mobahi, H., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553469 · Cited by 328 · cited 1×
“An especially compelling result was presented in Taylor et al. [28] where a system was trained that could recognize similar human poses while being highly invariant to other distractors, including changes in lighting.”
BackgroundWinder, S., Hua, G., & Brown, M. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206839 · Cited by 253
Experience-based navigation for long-term localisation
Churchill, W., & Newman, P. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913499193 · Cited by 225
Discriminant Embedding for Local Image Descriptors
Hua, G., Brown, M., & Winder, S. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408857 · Cited by 175
Boosting Binary Keypoint Descriptors
Trzcinski, T., Christoudias, M., Fua, P., & Lepetit, V. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.370 · Cited by 174
Learning task-specific similarity
Shakhnarovich, G. (2005) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 161
Descriptor Learning for Efficient Retrieval
Philbin, J. F., Isard, M., Šivic, J., & Zisserman, A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15558-1_49 · Cited by 160
Mikulík, A., Perďoch, M., Chum, O., & Matas, J. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15558-1_1 · Cited by 142
Konolige, K., & Bowman, J. (2009) · 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2009.5354121 · Cited by 138
Deep Learning of Invariant Features via Simulated Fixations in Video
Zou, W. Y., Shenghuo Zhu, B., Yu, K., & Ng, A. Y. (2012) · neural information processing systems · Cited by 129
Appearance change prediction for long-term navigation across seasons
Neubert, P., Sunderhauf, N., & Protzel, P. (2013) · 2013 European Conference on Mobile Robots · doi:10.1109/ecmr.2013.6698842 · Cited by 105
Feature Co-occurrence Maps: Appearance-based localisation throughout the day
Johns, E., & Yang, G.-Z. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631024 · Cited by 85
Slow, Decorrelated Features for Pretraining Complex Cell-like Networks
Bengio, Y., & Bergstra, J. (2009) · neural information processing systems · Cited by 75
How to learn an illumination robust image feature for place recognition
Lategahn, H., Beck, J., Kitt, B., & Stiller, C. (2013) · 2013 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2013.6629483 · Cited by 66
Towards illumination invariance for visual localization
Ranganathan, A., Matsumoto, S., & Ilstrup, D. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631110 · Cited by 62
Learning invariance through imitation
Taylor, G. W., Spiro, I., Bregler, C., & Fergus, R. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995538 · Cited by 45
Unsupervised learning of invariant features using video
Stavens, D., & Thrun, S. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539773 · Cited by 44
Generative Methods for Long-Term Place Recognition in Dynamic Scenes
Johns, E., & Yang, G. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0648-6 · Cited by 39
Task Specific Local Region Matching
Babenko, B., Dollár, P., & Belongie, S. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408848 · Cited by 32
Towards condition-invariant, top-down visual place recognition
Milford, M. J., Vig, E., Scheirer, W. J., & Cox, D. (2013) · QUT ePrints (Queensland University of Technology) · Cited by 6
Research works printed but not linked: 2
- [38] M. Brown, S. Winder, and G. Hua., 'Learning local image descriptors data,' http://www.cs.ubc.ca/ ∼ mbrown/patchdata/patchdata.html, 2011. -
- [9] M. Milford, E. Vig, W. Scheirer, and D. Cox, 'Towards conditioninvariant, top-down visual place recognition,' in Australasian Conference on Robotics and Automation , Sydney, Australia, Dec. 2013, pp. 1-10. -
Cited by
A Survey of Autonomous Driving: Common Practices and Emerging Technologies
Yurtsever, E., Lambert, J., Carballo, A., & Takeda, K. (2020) · IEEE Access · doi:10.1109/access.2020.2983149 · Cited by 1,889
“Both attempting to extract lighting invariant features [149] and assessing the quality of features [150] have been proposed.”
BackgroundVisual Place Recognition: A Survey
Lowry, S., Sünderhauf, N., Newman, P., Leonard, J. J., Cox, D., Corke, P., & Milford, M. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2496823 · Cited by 1,079
“Using webcam footage, Carlevaris-Bianco and Eustice [213]”
University of Michigan North Campus long-term vision and lidar dataset
Carlevaris-Bianco, N., Ushani, A. K., & Eustice, R. M. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364915614638 · Cited by 576
Self-citationStreet-view change detection with deconvolutional networks
Alcantarilla, P. F., Stent, S., Ros, G., Arroyo, R., & Gherardi, R. (2018) · Autonomous Robots · doi:10.1007/s10514-018-9734-5 · Cited by 320
A survey on Visual-Based Localization: On the benefit of heterogeneous data
Piasco, N., Sidibé, D., Demonceaux, C., & Gouet-Brunet, V. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.09.013 · Cited by 236
Metadata
- Authors
- Carlevaris-Bianco, N., & Eustice, R. M.
- Affiliations
- University of Michigan (US)
- ORCID
- Eustice, Ryan M.
- Venue
- 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems
- Publisher
- IEEE
- Published
- 2014-09
- Pages
- 2769-2776
- ISSN
- 2153-0858 (ISSN-L)
- OpenAlex
- W2052742500
- Cited by
- 75
- References
- 42
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 3 · 2023: 1 · 2022: 4 · 2021: 6 · 2020: 9 · 2019: 7 · 2018: 11 · 2017: 6 · 2016: 10 · 2015: 12 · 2014: 1
- FWCI
- 5.20
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- illumination robustness, feature-based image registration, feature point tracking, time-lapse video, descriptor learning, image patch matching
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.