Deep Visual MPC-Policy Learning for Navigation
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Humans can routinely follow a trajectory defined by a list of images/landmarks. However, traditional robot navigation methods require accurate mapping of the environment, localization, and planning. Moreover, these methods are sensitive to subtle changes in the environment. In this letter, we propose PoliNet, a deep visual model predictive control-policy learning method that can perform visual navigation while avoiding collisions with unseen objects on the navigation path. PoliNet takes in as input a visual trajectory and 360° images from robot's current view and outputs velocity commands fora planning horizon of N steps that optimally balance between trajectory following and obstacle avoidance. PoliNet is trained using a differentiable neural image predictive model and a traversability estimation model in an model predictive control setup, with minimal human supervision. PoliNet can be applied to visual trajectory in new scenes without retraining. We show experimentally that the robot can follow a visual trajectory even if it does not start from the exact same position and in the presence of previously unseen obstacles. We validated our algorithm with tests both in a realistic simulation environment and in the real world outperforming state-of-the-art baselines under similar conditions in success rate, coverage rate of the trajectory, and with lower computational load. We also show that we can generate visual trajectory in simulation and execute the corresponding path in the real environment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Simultaneous localization and mapping: part I
Durrant-Whyte, H., & Bailey, T. (2006) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2006.1638022 · Cited by 4,199
Matterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533 · cited 4×
“More recent approaches [4, 6, 7] have posed the navigation problem as a reinforcement learning task.”
BackgroundMethodInfluentialLearning agile and dynamic motor skills for legged robots
Hwangbo, J., Lee, J., Dosovitskiy, A., Bellicoso, D., Tsounis, V., Koltun, V., & Hutter, M. (2019) · Science Robotics · doi:10.1126/scirobotics.aau5872 · Cited by 1,482
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187
Gibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763
Self-citationDeep visual foresight for planning robot motion
Finn, C., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989324 · Cited by 599 · cited 6×
“Stochastic optimization [6]: We use a cross-entropy method (CEM) stochastic optimization algorithm similar to [6] as baseline of a different optimization approach.”
BackgroundMethodInfluentialLearning deep control policies for autonomous aerial vehicles with MPC-guided policy search
Zhang, T., Kahn, G., Levine, S., & Abbeel, P. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487175 · Cited by 452
Cited by
BADGR: An Autonomous Self-Supervised Learning-Based Navigation System
Kahn, G., Abbeel, P., & Levine, S. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3057023 · Cited by 284
Interactive Gibson Benchmark: A Benchmark for Interactive Navigation in Cluttered Environments
Xia, F., Shen, W. B., Li, C., Kasimbeg, P., Tchapmi, M. E., Toshev, A., Martin-Martin, R., & Savarese, S. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2965078 · Cited by 171
“They facilitated the development and benchmarking of vision-based (learned) navigation algorithms, and some even enabled to deploy on real robots the algorithms developed in simulation [12], [13].”
BackgroundSelf-citationMorales, E. F., Murrieta-Cid, R., Becerra, I., & Esquivel-Basaldua, M. A. (2021) · Intelligent Service Robotics · doi:10.1007/s11370-021-00398-z · Cited by 101
GNM: A General Navigation Model to Drive Any Robot
Shah, D., Sridhar, A., Bhorkar, A., Hirose, N., & Levine, S. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10161227 · Cited by 85
“Following a large body of research in visual navigation [27–31], we use a combination of topological graphs for high-level planning and image-goal policies for low-level control, which gives us an efficient way to scale reactive policies for long-range navigation [30, 32].”
BackgroundMethodInfluentialSelf-citationSrivastava, S., Li, C., Lingelbach, M., Martín-Martín, R., Xia, F., Vainio, K., Lian, Z., Gokmen, C., Buch, S., Liu, C. K., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.03332 · Cited by 55
Self-citation
Metadata
- Authors
- Hirose, N., Xia, F., Martin-Martin, R., Sadeghian, A., & Savarese, S.
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2019-10
- Volume
- 4
- Issue
- 4
- Pages
- 3184-3191
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W2963318716
- Cited by
- 81
- References
- 8
- FWCI
- 5.64
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep Visual MPC-Policy Learning for Navigation. https://app.cochat.ai/papers/10.1109/lra.2019.2925731
- The paper
- Hirose, N., Xia, F., Martin-Martin, R., Sadeghian, A., & Savarese, S. (2019). Deep Visual MPC-Policy Learning for Navigation. IEEE Robotics and Automation Letters. https://doi.org/10.1109/lra.2019.2925731
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.