A ToF-Camera as a 3D Vision Sensor for Autonomous Mobile Robotics
Open accessThe DOI matches its Crossref record; 35 of the 36 references it cites were found in a registry or an index.
Abstract
The aim of this paper is to deploy a time-of-flight (ToF) based photonic mixer device (PMD) camera on an Autonomous Ground Vehicle (AGV) whose overall target is to traverse from one point to another in hazardous and hostile environments employing obstacle avoidance without human intervention. The hypothesized approach of applying a ToF Camera for an AGV is a suitable approach to autonomous robotics because, as the ToF camera can provide three-dimensional (3D) information at a low computational cost, it is utilized to extract information about obstacles after their calibration and ground testing and is mounted and integrated with the Pioneer mobile robot. The workspace is a two-dimensional (2D) world map which has been divided into a grid/cells, where the collision-free path defined by the graph search algorithm is a sequence of cells the AGV can traverse to reach the target. PMD depth data is used to populate traversable areas and obstacles by representing a grid/cells of suitable size. These camera data are converted into Cartesian coordinates for entry into a workspace grid map. A more optimal camera mounting angle is needed and adopted by analysing the camera's performance discrepancy, such as pixel detection, the detection rate and the maximum perceived distances, and infrared (IR) scattering with respect to the ground surface. This mounting angle is recommended to be half the vertical field-of-view (FoV) of the PMD camera. A series of still and moving tests are conducted on the AGV to verify correct sensor operations, which show that the postulated application of the ToF camera in the AGV is not straightforward. Later, to stabilize the moving PMD camera and to detect obstacles, a tracking feature detection algorithm and the scene flow technique are implemented to perform a real-time experiment.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Shi, J., & Tomasi (1994) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR-94 · doi:10.1109/cvpr.1994.323794 · Cited by 7,724 · cited 2×
“The ‘Good Features to Track’ feature detection algorithm [50] is used to stabilize the moving AGV by comparing two consecutive frames.”
MethodRGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments
Henry, P., Krainin, M., Herbst, E., Ren, X., & Fox, D. (2012) · The International Journal of Robotics Research · doi:10.1177/0278364911434148 · Cited by 1,161
International Journal of Advanced Robotic Systems
(2017) · International Journal of Advanced Robotic Systems · doi:10.5772/1 · Cited by 634
Everett, H. R. (1995) · A K Peters/CRC Press eBooks · doi:10.1201/9781439863480 · Cited by 334
3D shape scanning with a time-of-flight camera
Cui, Y., Schuon, S., Chan, D., Thrun, S., & Theobalt, C. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540082 · Cited by 293
Artificial vision in road vehicles
Bertozzi, M., Broggi, A., Cellario, M., Fascioli, A., Lombardi, P., & Porta, M. (2002) · Proceedings of the IEEE · doi:10.1109/jproc.2002.801444 · Cited by 251 · cited 2×
“Over the last few decades, many different types of sensors [2] have been developed in the context of AGV pathplanning to avoid obstacles [3] such as infrared sensors [4], ultrasonic sensors, sonar [5], LADAR [6], laser rangefinders [7], camera data fused with radar [8] and stereo cameras with a projector [9].”
BackgroundInfluentialLateral and Depth Calibration of PMD-Distance Sensors
Lindner, M., & Kolb, A. (2006) · Lecture notes in computer science · doi:10.1007/11919629_53 · Cited by 131 · cited 1×
“Basically, the intrinsic parameters of the camera provide the transformation between the image coordinates and the pixel coordinates of the camera [44, 45].”
Background3D time-of-flight cameras for mobile robotics
May, S., Werner, B., Surmann, H., & Pervölz, K. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.281670 · Cited by 104 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundA Performance Review of 3D TOF Vision Systems in Comparison to Stereo Vision Systems
Hussmann, S. H., Ringbeck, T., & Hagebeuker, B. (2008) · InTech eBooks · doi:10.5772/5898 · Cited by 81 · cited 1×
“Hence, it is rotation-, translation- and illumination-invariant [10].”
Robust real-time 3D respiratory motion detection using time-of-flight cameras
Penne, J., Schaller, C., Joachim Hornegger, A., & Kuwert, T. (2008) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-008-0245-2 · Cited by 75 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundEstimating human 3D pose from Time-of-Flight images based on geodesic distances and optical flow
Schwarz, L., Mkhitaryan, A., Mateus, D., & Navab, N. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771333 · Cited by 71 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundView-invariant gesture recognition using 3D optical flow and harmonic motion context
Holte, M. B., Moeslund, T. B., & Fihl, P. (2010) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2010.07.012 · Cited by 68 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundOn-the-fly scene acquisition with a handy multi-sensor system
Huhle, B., Jenke, P., & Straßer, W. (2008) · International Journal of Intelligent Systems Technologies and Applications · doi:10.1504/ijista.2008.021288 · Cited by 55 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundShape and motion-based pedestrian detection in infrared images: a multi sensor approach
Fardi, B., Schuenert, U., & Wanielik, G. (2005) · IEEE Proceedings. Intelligent Vehicles Symposium, 2005. · doi:10.1109/ivs.2005.1505071 · Cited by 47
A Virtual Keyboard Based on True-3D Optical Ranging
Du, H., Oggier, T., Lustenberger, F., & Edoardo Charbon, A. (2005) · British Machine Vision Conference (BMVC) · doi:10.5244/c.19.27 · Cited by 46 · cited 3×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundApplications of a 3D Range Camera Towards Healthcare Mobility Aids
Bostelman, R. V., Russo, P., Albus, J. S., Hong, T., & Madhavan, R. (2006) · IEEE International Conference on Networking, Sensing and Control (ICNSC) · doi:10.1109/icnsc.2006.1673182 · Cited by 36 · cited 3×
“The ToF camera has been used to detect the standard size of obstacles and it can be used for obstacle and travel path detection applications for the blind and visually impaired by combining 3D range data with stereo audio feedback [21], using the algorithm to segment obstacles according to intensity and the 3D structu…”
BackgroundMethodA Taxonomy of Vision Systems for Ground Mobile Robots
Martínez-Gómez, J., Fernández‐Caballero, A., García-Varea, I., Rodriguez, L., & Romero-González, C. (2014) · International Journal of Advanced Robotic Systems · doi:10.5772/58900 · Cited by 35 · cited 1×
“Over the last few decades, many different types of sensors [2] have been developed in the context of AGV pathplanning to avoid obstacles [3] such as infrared sensors [4], ultrasonic sensors, sonar [5], LADAR [6], laser rangefinders [7], camera data fused with radar [8] and stereo cameras with a projector [9].”
BackgroundLaser-based navigation enhanced with 3D time-of-flight data
Yuan, F., Swadzba, A., Philippsen, R., Engin, O., Hanheide, M., & Wachsmuth, S. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2009.5152533 · Cited by 35 · cited 1×
“In addition, ToF cameras have been applied in robotic applications for obstacle avoidance [14, 15].”
MethodTowards Semantic Scene Analysis with Time-of-Flight Cameras
Holz, D., Schnabel, R., Droeschel, D., Stückler, J., & Behnke, S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-20217-9_11 · Cited by 33 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundManifold Learning for ToF-based Human Body Tracking and Activity Recognition
Schwarz, L., Mateus, D., Castañeda, V., & Navab, N. (2010) · British Machine Vision Conference (BMVC) · doi:10.5244/c.24.80 · Cited by 32 · cited 3×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundMobile robot map building from time-of-flight camera
Almansa-Valverde, S., Castillo, J. C., & Fernández‐Caballero, A. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.02.006 · Cited by 32 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundSLAM combining ToF and high-resolution cameras
Castañeda, V., Mateus, D., & Navab, N. (2011) · Proceedings · doi:10.1109/wacv.2011.5711569 · Cited by 32 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundMixIn3D: 3D Mixed Reality with ToF-Camera
Koch, R., Schiller, I., Bartczak, B., Kellner, F., & Köser, K. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-03778-8_10 · Cited by 31 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundAnalysis and characterization of the PMD camera for application in mobile robotics
Wiedemann, M., Sauer, M., Driewer, F., & Schilling, K. (2008) · IFAC Proceedings Volumes · doi:10.3182/20080706-5-kr-1001.02318 · Cited by 30 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundHypothesis based vehicle detection for increased simplicity in multi-sensor ACC
Alefs, B. G., Schreiber, D., & Clabian, M. (2005) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2005.1505112 · Cited by 25 · cited 1×
“Over the last few decades, many different types of sensors [2] have been developed in the context of AGV pathplanning to avoid obstacles [3] such as infrared sensors [4], ultrasonic sensors, sonar [5], LADAR [6], laser rangefinders [7], camera data fused with radar [8] and stereo cameras with a projector [9].”
BackgroundMoving obstacles detection and tracking with laser range finder
Rebai, K., Benabderrahmane, A., Azouaoui, O., & Ouadah, N. (2009) · International Conference on Advanced Robotics (ICAR) · Cited by 21
Localized qualitative navigation for indoor environments
Sgouros, N. M., Papakonstantinou, G., & Tsanakas, P. D. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1996.503890 · Cited by 21
Wide range Time of Flight camera for outdoor surveillance
Falie, D., & Buzuloiu, V. V. (2008) · Microwaves, Radar and Remote Sensing Symposium · doi:10.1109/mrrs.2008.4669550 · Cited by 19 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundRobot Cognition and Navigation: An Experiment with Mobile Robots
Patnaik, S. (2007) · doi:10.1007/978-3-540-68916-4 · Cited by 15 · cited 1×
“Hence, the perception sensor becomes the key sensory device for perceiving the environment in intelligent mobile robots, and the perception objective depends on three basic system qualities, namely rapidity, compactness and robustness [1].”
BackgroundFast 3D model acquisition from stereo images
Morency, L., Rahimi, A., & Darrell, T. (2003) · International Symposium on 3D Data Processing, Visualization and Transmission · doi:10.1109/tdpvt.2002.1024057 · Cited by 13 · cited 2×
“The authors in [36] used stereo cameras to identify the 3D orientation of the object and ToF cameras have been used for 3D object scanning, employing different approaches, such as passive image-based and super-resolution methods with probabilistic scan alignment [37].”
MethodRobust Ego-Motion Estimation with ToF Cameras.
Droeschel, D., May, S., Holz, D., Plöger, P.-G., & Behnke, S. (2009) · Cited by 13
3D range imaging for urban search and rescue robotics research
Bostelman, R. V., Hong, T. H., Madhavan, R., & Weiss, B. A. (2005) · IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) · doi:10.1109/ssrr.2005.1501254 · Cited by 12 · cited 1×
“There are many other applications in various fields [16] that have gained substantial research interest following the advent of the range sensor, such as robotic and machine vision in the field of mobile robotics search and rescue [17, 18], path-planning for manipulators [19], the acquisition of 3D scene geometry [20]…”
BackgroundVision based defect detection on 3D objects and path planning for processing
Weyrich, M., Klein, P., Laurowski, M., & Wang, Y. (2011) · Cited by 8
Reconstructing the geometry of an object using 3D TOF Camera
Francis, S. L. X., Anavatti, S. G., & Garratt, M. (2011) · doi:10.1109/mfcist.2011.5949516 · Cited by 8 · cited 1×
“As the dimensions of the image frame depend upon its distance from the camera and the camera’s field of view, an object’s height and width can be calculated using its pixel elements [47].”
MethodSelf-citationChen, S., Mastorakis, N. E., Rivas, F., & Mladenov, V. (2011) · Cited by 3
Research works printed but not linked: 13
- 2007, Mats Ahlskog. 3d vision
- May Stefan, 2009, 3D Time-of-Flight Ranging for Robotic Perception in Dynamic Environments
- Ringbeck T., 2007, A 3D Time of Flight Camera for Object Detection
- Weingarten J.W., 2004, In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 3, 2155
- Hansard M., 2012, SpringerBriefs in Computer Science
- Soutschek S., 2008, Computer Vision and Pattern Recognition Workshops, 1
- Distante C., 2010, Annals of the BMVA, London, 3, 1
- Teizer J., 2008, Sensors in Construction and Infrastructure Management, 103
- Conro R., Shape and Deformation Measurementusing Heterodyne Range Imaging Technology. Proceedings of 12th Asia-Pacific Conference on NDT, Auckland, New Zealand, November 2006.
- Kahlmann T., 2006, International Archives of the Photogrammetry, Remote Sensing, and Geoinformation Sciences, 136
- Möller T., 2005, Robust 3D Measurement with PMD Sensors
- Ramanandan S K, 2011, 3D ToF Camera Calibration and Image Pre-processing
- Jung Boyoon and Sukhatme Gaurav S. Detecting moving objects using a single camera on a mobile robot in an outdoor environment. In in International Conference on Intelligent Autonomous Systems, pages 980–987, 2004.
Cited by
Recent Advances in 3D Data Acquisition and Processing by Time-of-Flight Camera
He, Y. H., & Chen, S. (2019) · IEEE Access · doi:10.1109/access.2019.2891693 · Cited by 125
“The real-time 3D data can help robots to accomplish autonomous path-planning [57], [202].”
BackgroundComparative analysis of range sensors for the robust autonomous navigation – a review
Singh, R., & Nagla, K. S. (2019) · Sensor Review · doi:10.1108/sr-01-2019-0029 · Cited by 47
Exploring Autonomous Load-Carrying Mobile Robots in Indoor Settings: A Comprehensive Review
Leong, P. Y., & Ahmad, N. S. (2024) · IEEE Access · doi:10.1109/access.2024.3435689 · Cited by 42
Altuntaş, C. (2023) · Applied Sciences · doi:10.3390/app13116488 · Cited by 30
Kuang, B., Gu, C., Rana, Z. A., Zhao, Y., Sun, S., & Nnabuife, S. G. (2022) · Sensors · doi:10.3390/s22218393 · Cited by 15
“Active sensors (such as radar, laser scanner [17], structured light [18], time-of-flight (ToF) [19], etc.) are sensitive to environmental changes, heavy-weighted, have high energy consumption, and large-sized [16,20,21].”
Metadata
- Authors
- Francis, S. L. X., Anavatti, S. G., Garratt, M., & Shim, H.
- Affiliations
- UNSW Canberra (AU)
- Seoul National University (KR)
- Corresponding author
- Francis, Sobers Lourdu Xavier
- Venue
- International Journal of Advanced Robotic Systems
- Publisher
- SAGE Publications
- Published
- 2015-01-01
- Volume
- 12
- Issue
- 11
- ISSN
- 1729-8806 (ISSN-L)
- 1729-8814
- OpenAlex
- W1901506754
- Semantic Scholar
- dee5f977e0ae222f3c39c01f1f3e9179a8da38b8
- Cited by
- 25
- References
- 36
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 5 · 2023: 3 · 2022: 4 · 2020: 2 · 2019: 4 · 2018: 2 · 2017: 1
- FWCI
- 1.38
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- International Journal of Advanced Robotic Systems (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Contact-less Assessment of In-vivo Body Signals Using Microwave Doppler Radar (InTech) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Physics and Astronomy › Instrumentation › Advanced Optical Sensing Technologies
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- time-of-flight cameras, photonic mixer device, autonomous ground vehicles, obstacle avoidance, 3D vision sensor, grid map, infrared scattering, scene flow, feature detection, hazardous environments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.