Object Recognition in 3D Point Clouds Using Web Data and Domain Adaptation
The DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index.
Abstract
In recent years, object detection has become an increasingly active field of research in robotics. An important problem in object detection is the availability of a sufficient amount of labeled training data to learn good classifiers. In this paper we show how to significantly reduce the need for manually labeled training data by leveraging data sets available on the World Wide Web. Specifically, we show how to use objects from Google’s 3D Warehouse to train an object detection system for 3D point clouds collected by robots navigating through both urban and indoor environments. In order to deal with the different characteristics of the web data and the real robot data, we additionally use a small set of labeled point clouds and perform domain adaptation . Our experiments demonstrate that additional data taken from the 3D Warehouse along with our domain adaptation greatly improves the classification accuracy on real-world environments.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893 · cited 1×
“For the more complex outdoor scenes, we first bin the points into grid cells of size 25 25 25 cm3 and then run RANSAC plane fitting (Fischler and Bolles 1981) on each cell to find the surface orientations of each grid cell.”
MethodThe WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · cited 2×
“We used the implementation of LogitBoost in Weka (Hall et al., 2009) and the implementation of multi-class SVMs in LibSVM (Chang and Lin, 2001).”
MethodNormalized cuts and image segmentation
Shi, J., & Malik, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868688 · Cited by 15,502
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992 · cited 2×
“The next step fixes e at ke and optimizes Equation (2) to yield the new wk 1e : wk 1e argmin we i : e kei L wTe dei i e L wTe dei (4) When the loss function L is chosen to be the squared hinge loss function, this optimization yields standard support vector machine learning (Boser et al. 1992).”
BackgroundMethodMean shift: a robust approach toward feature space analysis
Comaniciu, D., & Meer, P. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000236 · Cited by 11,305 · cited 1×
“We experimented with the mean-shift clustering (Comaniciu and Meer 2002) and normalized cuts (Shi and Malik 2000) algorithms at various parameter settings and found that the former provided better segmentation.”
MethodLibsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,123
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884 · cited 2×
“Our experiments demonstrate that our two-domain adaptation methods lead to improvements over approaches without domain adaptation and alternatives such as LogitBoost (Friedman et al. 2000) and a regular multi-class SVM (Chang and Lin 2001).”
MethodCaruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · cited 1×
“Several relevant techniques have been developed for transfer learning (Caruana 1997 Dai et al. 2007) and, more recently, for domain adaptation (Chelba and Acero 2004 Daumé III and Marcu 2006 Jiang and Zhai 2007 Daumé III 2007).”
MethodLabelMe: A Database and Web-Based Tool for Image Annotation
Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0090-8 · Cited by 4,210 · cited 3×
“…through an urban environment will often observe cars and people from very close range and at angles different from those typically available in data sets such as LabelMe (Russell et al. 2008) furthermore, weather and lighting conditions could lead to considerable differences from webbased images.”
BackgroundUsing spin images for efficient object recognition in cluttered 3D scenes
Johnson, A. E., & Hebert, M. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.765655 · Cited by 2,599 · cited 6×
“For each point, we compute spin image features (Johnson and Hebert, 1999), which are 16 × 16 matrices describing the local shape around that point.”
BackgroundMethodInfluentialInternational Journal of Computer Vision
(2013) · International Journal of Computer Vision · doi:10.1007/11263.1573-1405 · Cited by 2,340
80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition
Torralba, A., Fergus, R., & Freeman, W. T. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.128 · Cited by 1,921
Boosting for transfer learning
Dai, W., Yang, Q., Xue, G.-R., & Yu, Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273521 · Cited by 1,724 · cited 2×
“Several relevant techniques have been developed for transfer learning (Caruana 1997 Dai et al. 2007) and, more recently, for domain adaptation (Chelba and Acero 2004 Daumé III and Marcu 2006 Jiang and Zhai 2007 Daumé III 2007).”
MethodShilane, P., Min, P., Kazhdan, M., & Funkhouser, T. (2004) · Proceedings Shape Modeling Applications, 2004. · doi:10.1109/smi.2004.1314504 · Cited by 1,419 · cited 2×
“A number of online shape databases have emerged in recent years, including the Princeton Shape Benchmark (Shilane et al. 2004) and Google’s 1019 3D Warehouse (Google 2008).”
MethodFrustratingly Easy Domain Adaptation
Daumé, H. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0907.1815 · Cited by 1,383 · cited 1×
“Daumé III (2007) argued that this approach works well since data points from the target domain have more influence than source domain points when making predictions about test data.”
BackgroundProceedings of the British Machine Vision Conference 2014
(2014) · doi:10.5244/c.28 · Cited by 1,337
Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2017)
Amendola, G., Ricca, F., & Truszczyński, M. (2017) · International Joint Conference on Artificial Intelligence · Cited by 1,179
Instance Weighting for Domain Adaptation in NLP
Jiang, J., & Zhai, C. (2007) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 952
Robotic Grasping of Novel Objects using Vision
Saxena, A., Driemeyer, J., & Ng, A. Y. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364907087172 · Cited by 923 · cited 1×
“In robotics, Saxena and colleagues (Saxena et al. 2008) used synthetically generated images of objects to learn grasp points for manipulation.”
BackgroundDomain Adaptation for Statistical Classifiers
Daume III, H., & Marcu, D. (2006) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1872 · Cited by 816
Scene completion using millions of photographs
Hays, J., & Efros, A. A. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276382 · Cited by 687
Multitask Learning: A Knowledge-Based Source of Inductive Bias
Caruana, R. A. (1993) · Machine Learning Proceedings 1993 · doi:10.1016/b978-1-55860-307-3.50012-5 · Cited by 679 · cited 2×
“Several relevant techniques have been developed for transfer learning (Caruana 1997 Dai et al. 2007) and, more recently, for domain adaptation (Chelba and Acero 2004 Daumé III and Marcu 2006 Jiang and Zhai 2007 Daumé III 2007).”
MethodScene completion using millions of photographs
Hays, J., & Efros, A. A. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276382 · Cited by 534
Discriminative Learning of Markov Random Fields for Segmentation of 3D Scan Data
Anguelov, D., Taskar, B., Chatalbashev, V., Koller, D., Gupta, D., Heitz, G., & Ng, A. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.133 · Cited by 356 · cited 2×
“Recently, several robotics research groups have also developed techniques for classification tasks based on visual and laser range information (Wellington et al., 2005; Anguelov et al., 2005; Douillard et al., 2008; Triebel et al., 2007; Sapp et al., 2008).”
BackgroundMethod3D zernike descriptors for content based shape retrieval
Novotni, M., & Klein, R. (2003) · Proceedings of the eighth ACM symposium on Solid modeling and applications · doi:10.1145/781606.781639 · Cited by 302 · cited 2×
“, 2003), 3D Zernike descriptors (Novotni and Klein, 2003), and spin image signatures (Johnson and Hebert, 1999; Assfalg et al.”
BackgroundMethodCorpus Variation and Parser Performance
Gildea, D. (2001) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 299
Improving Spatial Support for Objects via Multiple Segmentations
Malisiewicz, T., & Efros, A. A. (2007) · Procedings of the British Machine Vision Conference 2007 · doi:10.5244/c.21.55 · Cited by 280 · cited 4×
“In the context of vision-based recognition, (Malisiewicz and Efros, 2007) showed that it is beneficial to generate multiple possible segmentations of a scene, rather than relying on a single, possibly faulty segmentation.”
BackgroundInfluentialBuilding Support Vector Machines with Reduced Classifier Complexity
Keerthi, S. S., Chapelle, O., & DeCoste, D. (2006) · MPG.PuRe (Max Planck Society) · doi:10.5555/1248547.1248602 · Cited by 263
3D Shape Matching with 3D Shape Contexts
Körtgen, M., Park, G.-J., Novotni, M., & Klein, R. (2003) · Cited by 226
OPTIMOL: automatic Online Picture collecTion via Incremental MOdel Learning
Li, L.-J., Wang, G., & Fei-Fei, L. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383048 · Cited by 194
Adaptation of maximum entropy capitalizer: Little data can help a lot
Chelba, C., & Acero, A. (2006) · Computer Speech & Language · doi:10.1016/j.csl.2005.05.005 · Cited by 180 · cited 2×
“Several relevant techniques have been developed for transfer learning (Caruana 1997 Dai et al. 2007) and, more recently, for domain adaptation (Chelba and Acero 2004 Daumé III and Marcu 2006 Jiang and Zhai 2007 Daumé III 2007).”
BackgroundMethodUnsupervised language model adaptation
Bacchiani, M., & Roark, B. (2003) · 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). · doi:10.1109/icassp.2003.1198758 · Cited by 138 · cited 2×
“The problem of combining data from different sources is a major area of research in natural language processing (Hwa, 1999; Gildea, 2001; Bacchiani and Roark, 2003; Roark and Bacchiani, 2003; Chelba and Acero, 2004).”
Background2003 Shape Modeling International.
(2003) · doi:10.1109/smi.2003 · Cited by 135
Recognition by association via learning per-exemplar distances
Malisiewicz, T., & Efros, A. A. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587462 · Cited by 133 · cited 11×
“Malisiewicz and Efros (2008) showed that this threshold is not only natural, but also empirically gave good performance.”
BackgroundMethodInfluentialKonolige, K. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509796 · Cited by 127 · cited 2×
“Target domain exemplars are recorded by the prototype stereo camera and textured light projector system developed by Konolige (2010) (see Fig.”
MethodAdaptation of Maximum Entropy Capitalizer: Little Data Can Help a Lo.
Chelba, C. I., & Acero, A. (2004) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 121
Supervised and unsupervised PCFG adaptation to novel domains
Roark, B., & Bacchiani, M. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073472 · Cited by 92 · cited 2×
“The problem of combining data from different sources is a major area of research in natural language processing (Hwa, 1999; Gildea, 2001; Bacchiani and Roark, 2003; Roark and Bacchiani, 2003; Chelba and Acero, 2004).”
BackgroundContent-Based Retrieval of 3-D Objects Using Spin Image Signatures
Assfalg, J., Bertini, M., Bimbo, A. D., & Pala, P. (2007) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2006.886271 · Cited by 87 · cited 8×
“Shape retrieval methods have been proposed using a number of features including 3D shape contexts (Körtgen et al. 2003), 3D Zernike descriptors (Novotni and Klein 2003) and spin image signatures (Johnson and Hebert 1999 Assfalg et al. 2007).”
BackgroundMethodInfluentialSporleder, C., & Lapata, M. (2004) · The Association for Computational Linguistics · Cited by 73
Mei, Q., & Radev, D. (2016) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199573691.013.022 · Cited by 66
Supervised grammar induction using training data with limited constituent information
Hwa, R. (1999) · Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics - · doi:10.3115/1034678.1034699 · Cited by 65 · cited 2×
“The problem of combining data from different sources is a major area of research in natural language processing (Hwa, 1999; Gildea, 2001; Bacchiani and Roark, 2003; Roark and Bacchiani, 2003; Chelba and Acero, 2004).”
BackgroundInteracting Markov Random Fields for Simultaneous Terrain Modeling and Obstacle Detection
Wellington, C., Courville, A. C., & Stentz, A. (2005) · Robotics: Science and Systems · doi:10.15607/rss.2005.i.001 · Cited by 64 · cited 2×
“Recently, several robotics research groups have also developed techniques for classification tasks based on visual and laser range information (Wellington et al., 2005; Anguelov et al., 2005; Douillard et al., 2008; Triebel et al., 2007; Sapp et al., 2008).”
BackgroundMethodInstance-based AMN classification for improved object recognition in 2D and 3D laser range data
Triebel, R., Schmidt, R. K., Mozos, Ó. M., & Burgard, W. (2007) · Lincoln Repository (University of Lincoln) · Cited by 54
Laser and Vision Based Outdoor Object Mapping
Douillard, B., Fox, D., & Ramos, F. T. (2008) · Robotics: Science and Systems · doi:10.15607/rss.2008.iv.002 · Cited by 51 · cited 2×
“Recently, several robotics research groups have also developed techniques for classification tasks based on visual and laser range information (Wellington et al., 2005; Anguelov et al., 2005; Douillard et al., 2008; Triebel et al., 2007; Sapp et al., 2008).”
BackgroundMethodSelf-citationA fast data collection and augmentation procedure for object recognition
Sapp, B., Saxena, A., & Ng, A. Y. (2008) · National Conference on Artificial Intelligence · Cited by 38
Fast Probabilistic Labeling of City Maps
Posner, I., Cummins, M., & Newman, P. M. (2008) · Robotics: Science and Systems · doi:10.15607/rss.2008.iv.003 · Cited by 36 · cited 1×
“Newman’s group has also done classification in maps constructed using laser range and camera data (Posner et al. 2008).”
BackgroundAutomatic Classification of Objects in 3D Laser Range Scans
Nüchter, A., Hartmut Surmann, A., & Hertzberg, J. (2004) · Cited by 32
Research works printed but not linked: 21
- Bacchiani, M., Proceedings of the International Conference on Acoustics
- Boser, B.E., COLT ’92: Proceedings of the 5th Annual Workshop on Computational Learning Theory
- Dai, W., Proceedings of the International Conference on Machine Learning (ICML)
- Daumé III, H., Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)
- Douillard, B., Proceedings of Robotics: Science and Systems (RSS)
- Google, 2008, 3D Warehouse
- Hwa, R., Proceedings of the 37th Annual Meeting of the ACL
- Jiang, J., Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)
- Keerthi, S.S., 2006, Journal of Machine Learning Research, 7, 1493
- Konolige, K., Proceedings of the IEEE International Conference on Robotics & Automation (ICRA)
- Körtgen, M., The 7th Central European Seminar on Computer Graphics
- Li, L.-J., IEEE Computer Vision and Pattern Recognition (CVPR)
- Malisiewicz, T., Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
- Novotni, M., SM ’03: Proceedings of the 8th ACM Symposium on Solid Modeling and Applications
- Nüchter, A., Proceedings of the 8th Conference on Intelligent Autonomous Systems
- Posner, I., Proceedings of Robotics: Science and Systems (RSS)
- Roark, B., NAACL ’03: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology
- Sapp, B., Proceedings of the National Conference on Artificial Intelligence (AAAI)
- Shi, J., 2000, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 22, 731
- Shilane, P., 2004, Shape Modeling International, 167
- Wellington, C., Proceedings of Robotics: Science and Systems (RSS)
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“sentations, which define a solid as a combination of atoms in a dictionary, have been considered in robotics and computer vision, with dictionary learned from data [266] or based on existing repositories of object models [149], [157].”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Semantic Scene Completion from a Single Depth Image
Song, S., Yu, F., Zeng, A., Chang, A. X., Savva, M., & Funkhouser, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.28 · Cited by 1,209
“One possible approach to obtain the complete geometry and semantic labels for a scene is to retrieve and fit instance-level 3D mesh models to the observed depth map [7, 30, 4, 16, 23, 17, 14].”
BackgroundDomain adaptation for object recognition: An unsupervised approach
Gopalan, R., Li, R., & Chellappa, R. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126344 · Cited by 1,053
“Some recent efforts focus on semi-supervised DA [33, 7, 26].”
BackgroundA Survey of Research on Cloud Robotics and Automation
Kehoe, B., Patil, S., Abbeel, P., & Goldberg, K. (2015) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2376492 · Cited by 833
“New approaches are required that are robust to dirty data.”
Background
Metadata
- Authors
- Lai, K., & Fox, D.
- Affiliations
- University of Washington (US)
- Corresponding author
- Lai, Kevin
- Fox, Dieter
- ORCID
- Fox, Dieter
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2010-05-04
- Volume
- 29
- Issue
- 8
- Pages
- 1019-1037
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W2129091674
- Cited by
- 152
- References
- 51
- Citations by year
- 2025: 2 · 2024: 1 · 2023: 4 · 2022: 4 · 2021: 3 · 2020: 13 · 2019: 9 · 2018: 5 · 2017: 13 · 2016: 17 · 2015: 19 · 2014: 12 · 2013: 19 · 2012: 18
- FWCI
- 16.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- The International Journal of Robotics Research (publishedVersion)
- Funders
- Defense Advanced Research Projects Agency
- Multidisciplinary University Research Initiative
- Natural Sciences and Engineering Research Council of Canada
- Office of Naval Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- object recognition, 3D point clouds, object detection, domain adaptation, web data, robot navigation, urban environment, indoor environment, training data reduction, point cloud classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.