Real-Time Visual SLAM for Autonomous Underwater Hull Inspection Using Visual Saliency
The DOI matches its Crossref record; 71 of the 73 references it cites were found in a registry or an index.
Abstract
This paper reports a real-time monocular visual simultaneous localization and mapping (SLAM) algorithm and results for its application in the area of autonomous underwater ship hull inspection. The proposed algorithm overcomes some of the specific challenges associated with underwater visual SLAM, namely, limited field of view imagery and feature-poor regions. It does so by exploiting our SLAM navigation prior within the image registration pipeline and by being selective about which imagery is considered informative in terms of our visual SLAM map. A novel online bag-of-words measure for intra and interimage saliency are introduced and are shown to be useful for image key-frame selection, information-gain-based link hypothesis, and novelty detection. Results from three real-world hull inspection experiments evaluate the overall approach, including one survey comprising a 3.4-h/2.7-km-long trajectory.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Speeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557 · cited 14×
“Due to the noise sensitivity of the gradient orientation calculation, we found that SIFT's descriptor tends to assign two similar texture patches as two distinct words, whereas SURF's wavelet descriptor tends to assign them to the same type of word.”
MethodInfluentialIntroduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Video Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367 · cited 4×
“This is in contrast with visual place recognition techniques, which typically have vocabulary sizes in the 4000– 11 000 range or more [50], [55]–[58].”
MethodResultInfluentialComputational modelling of visual attention
Itti, L., & Koch, C. (2001) · Nature Reviews Neuroscience · doi:10.1038/35058500 · Cited by 4,733
A STATISTICAL INTERPRETATION OF TERM SPECIFICITY AND ITS APPLICATION IN RETRIEVAL
SPARCK JONES, K. (1972) · Journal of Documentation · doi:10.1108/eb026526 · Cited by 4,505
Visual categorization with bags of keypoints
Csurka, G. (2004) · Cited by 4,379
Simultaneous localization and mapping: part I
Durrant-Whyte, H., & Bailey, T. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1638022 · Cited by 4,199
Simultaneous localization and mapping (SLAM): part II
Bailey, T., & Durrant-Whyte, H. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1678144 · Cited by 2,554
Textons, the elements of texture perception, and their interactions
Julesz, B. (1981) · Nature · doi:10.1038/290091a0 · Cited by 1,703 · cited 1×
“One of the original uses of BoW is for texture recognition [62], [63].”
BackgroundUnderstanding inverse document frequency: on theoretical arguments for IDF
Robertson, S. (2004) · Journal of Documentation · doi:10.1108/00220410410560582 · Cited by 1,541
Representing and Recognizing the Visual Appearance of Materials Using Three-Dimensional Textons
Leung, T., & Malik, J. (2001) · International Journal of Computer Vision · doi:10.1023/a:1011126920638 · Cited by 1,498 · cited 1×
“Originally developed for text-based applications, the general BoW approach was first adapted and expanded to images in [47], [49], and [50], allowing for aggregate con-”
MethodFAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance
Cummins, M., & Newman, P. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364908090961 · Cited by 1,472 · cited 4×
“This is in contrast with visual place recognition techniques, which typically have vocabulary sizes in the 4000– 11 000 range or more [50], [55]–[58].”
BackgroundMethodResultInfluentialGlobally Consistent Range Scan Alignment for Environment Mapping
Lu, F., & Milios, E. (1997) · Autonomous Robots · doi:10.1023/a:1008854305733 · Cited by 1,304
Saliency, Scale and Image Description
Kadir, T., & Brady, M. (2001) · International Journal of Computer Vision · doi:10.1023/a:1012460413855 · Cited by 1,195
iSAM: Incremental Smoothing and Mapping
Kaess, M., Ranganathan, A., & Dellaert, F. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2006706 · Cited by 1,076
Sampling Strategies for Bag-of-Features Image Classification
Nowak, E., Jurie, F., & Triggs, B. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744085_38 · Cited by 927 · cited 1×
“Alternatively to the above channel-based methods, several BoW saliency representations have recently been explored [45]–[48].”
MethodA Statistical Approach to Material Classification Using Image Patch Exemplars
Varma, M., & Zisserman, A. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.182 · Cited by 607
ON THE SPECIFICATION OF TERM VALUES IN AUTOMATIC INDEXING
SALTON, G., & YANG, C. S. (1973) · Journal of Documentation · doi:10.1108/eb026562 · Cited by 583
FrameSLAM: From Bundle Adjustment to Real-Time Visual Mapping
Konolige, K., & Agrawal, M. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2004832 · Cited by 532
Fast and Incremental Method for Loop-Closure Detection Using Bags of Visual Words
Angeli, A., Filliat, D., Doncieux, S., & Meyer, J.-A. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2004514 · Cited by 514
Near Duplicate Image Detection: min-Hash and tf-idf Weighting
Chum, O., Philbin, J., & Zisserman, A. (2008) · Procedings of the British Machine Vision Conference 2008 · doi:10.5244/c.22.50 · Cited by 460
On the burstiness of visual elements
Jegou, H., Douze, M., & Schmid, C. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206609 · Cited by 405
A framework for visual saliency detection with applications to image thumbnailing
Marchesotti, L., Cifarelli, C., & Csurka, G. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459467 · Cited by 322
Exactly Sparse Delayed-State Filters for View-Based SLAM
Eustice, R. M., Singh, H., & Leonard, J. J. (2006) · IEEE Transactions on Robotics · doi:10.1109/tro.2006.886264 · Cited by 306
Self-citationLCM: Lightweight Communications and Marshalling
Huang, A. S., Olson, E., & Moore, D. C. (2010) · 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2010.5649358 · Cited by 299
Highly scalable appearance-only SLAM - FAB-MAP 2.0
Cummins, M., & Newman, P. (2009) · Robotics: Science and Systems V · doi:10.15607/rss.2009.v.039 · Cited by 282
Advanced perception, navigation and planning for autonomous in-water ship hull inspection
Hover, F. S., Eustice, R. M., Kim, A., Englot, B., Johannsson, H., Kaess, M., & Leonard, J. J. (2012) · The International Journal of Robotics Research · doi:10.1177/0278364912461059 · Cited by 249
Self-citationGlobal A-Optimal Robot Exploration in SLAM
Sim, R., & Roy, N. (2006) · Proceedings of the 2005 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2005.1570193 · Cited by 238
Imaging sonar-aided navigation for autonomous underwater harbor surveillance
Johannsson, H., Kaess, M., Englot, B., Hover, F., & Leonard, J. (2010) · 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2010.5650831 · Cited by 193 · cited 2×
“[20] to work in real time and to perform automatic registration of sonar hull imagery.”
BackgroundMethodSemantics-Preserving Bag-of-Words Models and Applications
Wu, L., Hoi, S. C. H., & Yu, N. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2045169 · Cited by 191
Information-Based Compact Pose SLAM
Ila, V., Porta, J. M., & Andrade-Cetto, J. (2010) · IEEE Transactions on Robotics · doi:10.1109/tro.2009.2034435 · Cited by 183
Visually Augmented Navigation for Autonomous Underwater Vehicles
Eustice, R. M., Pizarro, O., & Singh, H. (2008) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2008.923547 · Cited by 182
Self-citationCovariance recovery from a square root information matrix for data association
Kaess, M., & Dellaert, F. (2009) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2009.06.008 · Cited by 160
An ROV Stereovision System for Ship-Hull Inspection
Negahdaripour, S., & Firoozfam, P. (2006) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2005.851391 · Cited by 159
Visual inspection of hydroelectric dams using an autonomous underwater vehicle
Ridao, P., Carreras, M., Ribas, D., & García, R. (2010) · Journal of Field Robotics · doi:10.1002/rob.20351 · Cited by 152
Konolige, K. (2004) · National Conference on Artificial Intelligence · Cited by 140
The Bayes Tree: An Algorithmic Foundation for Probabilistic Robot Mapping
Kaess, M., Ila, V., Roberts, R. B., & Dellaert, F. (2010) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-17452-0_10 · Cited by 120
Pose-graph visual SLAM with geometric model selection for autonomous underwater ship hull inspection
Kim, A., & Eustice, R. M. (2009) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2009.5354132 · Cited by 109
Self-citationPropagating Covariance in Computer Vision
Haralick, R. M. (2000) · Computational imaging and vision · doi:10.1007/978-94-015-9538-4_9 · Cited by 88
UWIT: underwater image toolbox for optical image processing and mosaicking in MATLAB
Eustice, R. M., Pizarro, O. R., Singh, H., & Howland, J. C. (2003) · IEEE Underwater Technology (UT) · doi:10.1109/ut.2002.1002415 · Cited by 86
Self-citationShip Hull Inspection with the HAUV: US Navy and NATO Demonstrations Results
Vaganay, J., Elkins, M., Esposito, D., O'Halloran, W., Hover, F. S., & Kokko, M. A. (2006) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.1109/oceans.2006.307039 · Cited by 79
SLAM for ship hull inspection using exactly sparse extended information filters
Walter, M., Hover, F., & Leonard, J. (2008) · 2008 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2008.4543408 · Cited by 78
Propagating covariance in computer vision
Haralick, R. M. (2002) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.1994.576335 · Cited by 72
Online and Incremental Appearance-based SLAM in Highly Dynamic Environments
Kawewong, A., Tongprasit, N., Tangruamsub, S., & Hasegawa, O. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364910371855 · Cited by 70
Online constraint network optimization for efficient maximum likelihood map learning
Grisetti, G., Lodi Rizzini, D., Stachniss, C., Olson, E., & Burgard, W. (2008) · 2008 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2008.4543481 · Cited by 69
Active control for single camera SLAM
Vidal-Calleja, T., Davison, A. J., Andrade-Cetto, J., & Murray, D. W. (2006) · Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006. · doi:10.1109/robot.2006.1641988 · Cited by 65
Unsupervised discovery of co-occurrence in sparse high dimensional data
Chum, O., & Matas, J. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539997 · Cited by 64
Larlus, D., Verbeek, J. J., & Jurie, F. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-009-0245-x · Cited by 58
A Bag of Words Approach for 3D Object Categorization
Toldo, R., Castellani, U., & Fusiello, A. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01811-4_11 · Cited by 55
Autonomous robots in the fog of war
Weiss, L. G. (2011) · IEEE Spectrum · doi:10.1109/mspec.2011.5960163 · Cited by 44
Spatially-Adaptive Learning Rates for Online Incremental SLAM
Olson, E., Leonard, J. J., & Teller, S. J. (2007) · Robotics: Science and Systems · doi:10.15607/rss.2007.iii.010 · Cited by 37
Menegaldo, L. L., Ferreira, G. A. N., Santos, M. F. D., & Guerato, R. S. (2009) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2009.2025716 · Cited by 37
Application of Locality Sensitive Hashing to realtime loop closure detection
Shahbazi, H., & Zhang, H. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6095099 · Cited by 30
de Carvalho, A. A., Sagrilo, L. V. S., Silva, I. C. F. G. D. A., Rebello, J. M. A., & Carneval, R. O. (2003) · Applied Ocean Research · doi:10.1016/j.apor.2004.02.004 · Cited by 28
Bryson, M., & Sukkarieh, S. (2005) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2005.1545114 · Cited by 28
Lamp Ray: ship hull assessment for value, safety and readiness
Harris, S. E., & Slate, E. V. (1999) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.1109/oceans.1999.799792 · Cited by 23
Ship berthing and hull inspection using the CetusII AUV and MIRIS high-resolution sonar
Trimble, G. M., & Belcher, E. O. (2004) · doi:10.1109/oceans.2002.1192132 · Cited by 20
Kim, A., & Eustice, R. M. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6094820 · Cited by 20
Self-citationOn-line visual vocabularies for robot navigation and mapping
Nicosevici, T., & García, R. (2009) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2009.5354392 · Cited by 20
SIRUS: A mobile robot for Floating Production Storage and Offloading (FPSO) ship hull inspection
Menegaldo, L. L., Santos, M. F., Ferreira, G. A. N., Guerato, R. S., & Moscato, L. A. (2008) · IEEE International Workshop on Advanced Motion Control (AMC) · doi:10.1109/amc.2008.4516036 · Cited by 18
Application of locality sensitive hashing to realtime loop closure detection
Shahbazi, H., & Zhang, H. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6048862 · Cited by 17
Active Visual SLAM with Exploration for Autonomous Underwater Navigation.
Kim, A. (2012) · Deep Blue (University of Michigan) · Cited by 16
Self-citationLee, Y.-J., & Song, J. (2010) · Advanced Robotics · doi:10.1163/016918610x512613 · Cited by 16
Performing ship hull inspections using a remotely operated vehicle
Lynn, D. C., & Bohlander, G. S. (1999) · doi:10.1109/oceans.1999.804763 · Cited by 13
Importance of Feature Locations in Bag-of-Words Image Classification
Lazic, N., & Aarabi, P. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.365989 · Cited by 7
Automated Underwater Hull Maintenance Vehicle
Bohlander, G. S., Hageman, G. R., Halliwell, F. S., Juers, R. H., & Lynn, D. C. (1992) · Defense Technical Information Center (DTIC) · Cited by 6
Underwater Inspection for Welding and Overhaul
Mittleman, J., & SWAN, L. (1993) · Naval Engineers Journal · doi:10.1111/j.1559-3584.1993.tb02755.x · Cited by 4
Spatially-Adaptive Learning Rates for Online Incremental SLAM
Olson, E., Leonard, J. L., & Teller, S. J. (2008) · The MIT Press eBooks · doi:10.7551/mitpress/7830.003.0011 · Cited by 4
UNDERWATER SHIP HULL INSPECTION
Mittleman, J., & Wyman, D. B. (1980) · Naval Engineers Journal · doi:10.1111/j.1559-3584.1980.tb05263.x · Cited by 3
Foreground Objects Recognition in Video Based on Bag-of-Words Model
Hu, M.-J., Li, C., Qu, Y., & Huang, J. (2009) · Chinese Conference on Pattern Recognition · doi:10.1109/ccpr.2009.5344073 · Cited by 3
Research works printed but not linked: 4
- kim, 2012, Next-best-view visual SLAM for bounded-error area coverage, presented at the IROS Workshop Active Semantic Percept
- vaganay, 2009, Hovering autonomous underwater vehicle—system design improvements and performance evaluation results, the 6th International Symposium on Unmanned Untethered Submersible Technology, 1
- olds, 2003, Marine mammals systems in support of force protection, SSC San Diego Biennial Review, 131
- johnson-roberson, 2010, Large-scale multi-sensor 3D reconstructions and visualizations of unstructured underwater environments
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
AUV Navigation and Localization: A Review
Paull, L., Saeedi, S., Seto, M., & Li, H. (2014) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2013.2278891 · Cited by 1,536
Autonomous Underwater Vehicle navigation: A review
Zhang, B., Ji, D., Liu, S., Zhu, X., & Xu, W. (2023) · Ocean Engineering · doi:10.1016/j.oceaneng.2023.113861 · Cited by 366
Progressively Complementarity-Aware Fusion Network for RGB-D Salient Object Detection
Chen, H., & Li, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00322 · Cited by 336
Metadata
- Authors
- Kim, A., & Eustice, R. M.
- Affiliations
- University of Michigan (US)
- Venue
- IEEE Transactions on Robotics
- Published
- 2013-06
- Volume
- 29
- Issue
- 3
- Pages
- 719-733
- ISSN
- 1552-3098 (ISSN-L)
- 1941-0468
- OpenAlex
- W2020950478
- Cited by
- 263
- References
- 73
- Citations by year
- 2026: 10 · 2025: 21 · 2024: 21 · 2023: 18 · 2022: 17 · 2021: 29 · 2020: 22 · 2019: 20 · 2018: 14 · 2017: 16 · 2016: 28 · 2015: 26 · 2014: 14 · 2013: 7
- FWCI
- 40.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Robotics (publishedVersion)
- Funders
- Massachusetts Institute of Technology
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Engineering › Ocean Engineering › Underwater Vehicles and Communication Systems
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- autonomous underwater vehicles, ship hull inspection, visual saliency, monocular SLAM, bag-of-words, key frame extraction, information gain, novelty detection, image registration, real-time SLAM
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.